-- Logs begin at Mon 2022-08-08 15:47:11 UTC, end at Mon 2022-08-08 17:50:27 UTC. -- Aug 08 16:39:27.867958 dv-cinder-852210-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 16:39:27.875057 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 16:39:27.877021 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 16:39:27 2022] *** Aug 08 16:39:27.877021 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 16:39:27.877021 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 16:39:27.877021 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: nodename: dv-cinder-852210-3-iscsi Aug 08 16:39:27.877021 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: machine: x86_64 Aug 08 16:39:27.877021 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: clock source: unix Aug 08 16:39:27.877021 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: pcre jit disabled Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: detected number of CPU cores: 4 Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: current working directory: / Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: detected binary path: /usr/bin/uwsgi-core Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: your processes number limit is 47680 Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: your memory page size is 4096 bytes Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: detected max file descriptor number: 2048 Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: lock engine: pthread robust mutexes Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: thunder lock: enabled Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 16:39:27.877485 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 16:39:27.917504 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: Python main interpreter initialized at 0x564e4e64f9c0 Aug 08 16:39:27.917504 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: python threads support enabled Aug 08 16:39:27.917504 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: your server socket listen backlog is limited to 100 connections Aug 08 16:39:27.917504 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: your mercy for graceful operations on workers is 90 seconds Aug 08 16:39:27.917894 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 16:39:27.917972 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: *** Operational MODE: preforking *** Aug 08 16:39:27.918246 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: *** uWSGI is running in multiple interpreter mode *** Aug 08 16:39:27.918246 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: spawned uWSGI master process (pid: 92247) Aug 08 16:39:27.918436 dv-cinder-852210-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 16:39:27.919399 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: spawned uWSGI worker 1 (pid: 92260, cores: 1) Aug 08 16:39:27.919399 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: spawned uWSGI worker 2 (pid: 92261, cores: 1) Aug 08 16:39:27.920795 dv-cinder-852210-3-iscsi devstack@keystone.service[92247]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 16:39:33.433651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 16:39:33.433651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: from cryptography.utils import int_from_bytes Aug 08 16:39:33.434494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 16:39:33.434494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: from cryptography.utils import int_from_bytes Aug 08 16:39:33.434622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 16:39:33.434622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: from cryptography.utils import int_from_bytes Aug 08 16:39:33.434622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 16:39:33.434622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: from cryptography.utils import int_from_bytes Aug 08 16:39:33.483944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 16:39:33.484103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 16:39:33.484185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 16:39:33.484273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 16:39:33.484350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] command line args: [] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 16:39:33.484426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] command line args: [] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 16:39:33.484584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 16:39:33.484671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 16:39:33.484754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ================================================================================ {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 16:39:33.484826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ================================================================================ {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 16:39:33.484992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] admin_token = **** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.485123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] admin_token = **** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.485237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] config_dir = [] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.485418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] config_dir = [] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.485507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.485585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] config_source = [] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.485660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.485762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] debug = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.485870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] config_source = [] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.485953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] debug = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.486025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.486235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.486382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.486492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.486601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.486704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.486799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.486900 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] list_limit = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] log_config_append = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] list_limit = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] log_config_append = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] log_dir = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487883 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.487985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] log_file = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.488080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] log_dir = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.488080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] log_file = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.488258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.488371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.488460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.488668 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.488745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.488911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.488998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.489114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.489242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.489328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.489405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.489528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.489609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.489715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.489884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.489976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.490181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.490264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.490410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.490493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.490641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.490718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.490886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.491002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.491130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.491282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.491410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.491572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.491863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.491939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.492068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] publish_errors = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.492259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.492343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.492490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.492664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.492824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.492902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.493018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] publish_errors = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.493127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.493273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.493349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] standard_threads = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.493553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.493632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.493735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.493842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.493981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.494065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.494137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] use_journal = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.494313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] use_json = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.494398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.494476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] use_stderr = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.494592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] standard_threads = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.494669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] use_syslog = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.494856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.494933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.495107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.495193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.495310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.495386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.495490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] use_journal = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.495574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.495718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] use_json = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.495796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.495868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] use_stderr = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.495991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] use_syslog = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.496065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.496258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 16:39:33.496346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.496550 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.496633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.496789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.496873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.497123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.497209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.497470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.497545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.497834 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.497916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.498146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.498236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.498461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.498535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.498766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.498878 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.499083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.499303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.499416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.499678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.499779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.499942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.500521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.500697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.500807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.500951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.501143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.501222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.501442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.501519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.501697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.501840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.501975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.502238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.502323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.502501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.502604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.502755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.502956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.503040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.503241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.503326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.503512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.503683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.503763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.504005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.504090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.504241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.504317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.504479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.504641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.504718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.504944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.505020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.505197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.505285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.505462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.505577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.505702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.505944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.506064 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.506198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.506299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.506435 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.506555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.506682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.506784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.506910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.506995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.507146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.507235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.507381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.507465 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.507612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.507719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.507842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.507930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.508073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.508152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.508300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.508376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.508534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.508633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.508774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.508887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.508988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.509105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.509222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.509312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.509451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.509537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.509688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.509773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.509947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.510028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.510187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.510271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.510414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.510493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.510648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.510729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.510881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.510960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.511114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.511196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.511347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.511429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.511580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.511659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.511807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.511887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.512044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.512187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.512258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.512318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.512496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.512584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.512725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.512843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.512974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.513052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.513205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.513283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.513434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.513512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.513668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.513747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.513927 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.514009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.514171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.514260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.514420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.514502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.514667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.514757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.514911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.514995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.515133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.515202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.515366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.515457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.515598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.516272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.516412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.516499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.516654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.516731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.516892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.516972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.517133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.517222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.517367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.517444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.517601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.517685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.517914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.518007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.518149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.518234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.518383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.518463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.518619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.518703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.518867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.518944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.519107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.519185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.519351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.519462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.519592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.519703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.519838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.519942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.520078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.520163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.520319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.520396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.520558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.520635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.520793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.520881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.521028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.521105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.521262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.521338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.521491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.521577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.521725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.521833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.521987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.522082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.522224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.522316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.522457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.522536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.522707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.522783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.522963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.523048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.523215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.523292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.523445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] auth.external = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.523524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.523674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.523760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] auth.external = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.523914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.523989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.524186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.524272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.524415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] auth.password = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.524494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.524643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] auth.token = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.524732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] auth.password = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.524880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.524964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] auth.token = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.525114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.525192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.525352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.525436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.525578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.525655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.525843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.525926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.526091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.526179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.526318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.526397 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.526551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] credential.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.526629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.526788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] credential.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.526874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.527031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.527109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.527278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.527357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.527511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.527598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.527752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.527830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.527989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.528067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.528228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.528306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.528490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.528573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.528724 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.528802 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.528973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.529053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.529229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.529306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.529483 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.529560 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.529742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.529837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.530017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.530097 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.530266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.530349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.530497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] federation.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.530576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.530730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.530817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] federation.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.530974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.531059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.531217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.531323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.531498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.531576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.531733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.531813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.531984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.532064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.532225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.532303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.532456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.532533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.532709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.532789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.532944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.533023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.533188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.533266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.533420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.533500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.533662 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.533738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.533929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.534010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.534164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.534244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.534395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.534474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.534630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.534706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.534883 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.534969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.535119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.535197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.535360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.535438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.535614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.535706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.535868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.535947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.536103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.536189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.536330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.536409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.536560 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.536677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.536806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.536883 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.537051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.537132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.537301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.537381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.537541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.537619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.537812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.537895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.538059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.538138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.538293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.538383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.538522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.538601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.538771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.538858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.538980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.539069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.539211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.539289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.539445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.539530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.539680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.539767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.539920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.539997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.540146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.540230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.540393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.540481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.540639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.540718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.540872 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.540951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.541115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.541201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.541360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.541437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.541590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.541668 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.541845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.541939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.542064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.542141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.542304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.542384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.542539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.542626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.542775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.542853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.543006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.543090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.543239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.543326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.543477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.543554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.543719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.543796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.543949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.544035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.544176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.544255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.544411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.544488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.544654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.544739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.544888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.544966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.545121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.545200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.545354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.545440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.545582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.545660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.545834 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.545944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.546082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.546186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.546307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.546385 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.546551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.546630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.546795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.546886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.547065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.547144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.547303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.547403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.547531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.547632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.547762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.547839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.547991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.548068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.548225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.548310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.548449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.548526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.548685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.548761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.548925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.549010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.549168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.549246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.549411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.549489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.549655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.549742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.549922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.550002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.550156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.550233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.550408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.550499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.550650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.550729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.550890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.550968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.551130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.551216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.551372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.551454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.551622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.551701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.551864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.551948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.552103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.552180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.552335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.552412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.552577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.552658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.552811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.552890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.553070 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.553160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.553273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.553353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.553510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.553587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.553756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.553854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.554016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.554098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.554251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.554328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.554487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] resource.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.554564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.554729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] resource.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.554808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.554973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.555051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.555208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.555286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.555450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.555528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.555697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.555773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.555931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.556009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.556176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.556256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.556411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.556489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.556647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.556725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.556880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] role.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.556960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.557117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] role.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.557194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] role.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.557344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.557420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] role.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.557589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.557666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.557865 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.557960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.558112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.558197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.558359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.558440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.558604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.558682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.558846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.558925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.559090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.559169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.559334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.559413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.559564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.559642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.559809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.559888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.560053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.560130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.560299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.560378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.560545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.560625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.560795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.560934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.561168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.561255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.561418 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.561498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.561666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.561744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.561957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.562047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.562284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.562361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.563037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.563123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.563300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.563388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.563554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.563633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.563794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.563872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.564025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.564111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.564257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.564336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.564487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.564566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.564725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.564810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.564978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.565055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.565224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.565301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.565458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.565545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.565691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.565770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.565945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.566025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.566185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.566305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] token.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.566428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] token.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.566543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.566671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.566790 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.566926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.567012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.567169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.567269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.567432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.567512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.567687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.567773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.567930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.568009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.568178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.568256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.568426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.568512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.568660 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.568785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.568897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.568976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.569128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.569215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.569364 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.569440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.569608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.569685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.569859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.569955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.570099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.570179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.570346 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.570422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.570632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.connection = **** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.570723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.570874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.570955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.connection = **** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.571118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.571207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.571353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.571432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.571591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.571679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.571824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.571902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.572056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.572136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.572287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.572439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.572515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.572592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.572750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.572828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.572979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.573064 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.573218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.573297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.573449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.573528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.573687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.573772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.573942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.574019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.574183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.574261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.574409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.574499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.574648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.574724 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.574875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.574952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.575044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 16:39:33.575131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 16:39:33.575272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 16:39:33.605056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.605281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.605370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.605469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.605550 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.605667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.615901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.616076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.644480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.644706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.644793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.644993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.696778 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.697250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.746570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.746803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.747552 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.747794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.810565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.810791 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.810986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.811209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.811448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.811648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.811749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.812108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.813090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.813473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.813881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.814232 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.835591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.835701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.835830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.835914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.836061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.836144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.836281 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.836362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.836498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.836579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.836709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.836792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.836920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.837002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.847980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.848325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.861080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.861315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.861542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.861770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.862011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.862229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.862431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.862515 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.862662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.862886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.863110 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.863327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.863541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.863753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 16:39:33.879777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.879893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.879972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 16:39:33.880045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 16:39:33.880323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.880406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.880527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 16:39:33.880608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 16:39:33.880806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.880887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.881063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.881143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.881323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.881405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.881579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.881660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.892884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.892994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.893156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.893239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.893426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.893508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.893703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.893818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.900567 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.900658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.903707 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.903799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.903988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.904072 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.904268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.904350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.904546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.904637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.904827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.904909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.905086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.905168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.905354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.905443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.905621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.905700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.905912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.905996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.906187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.906301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.906480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.906563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.929525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.929722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.931025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.931241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.931324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.931528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.935841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.936126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.936209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.936406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.936488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.936766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.943549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.943832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.943958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.944248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.947933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.948182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.948376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.948501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.948671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.948753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.948956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.949046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.949242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.949325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.949522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.949603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.949814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.949910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.950114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.950402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.965865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.966141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.966521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.966804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.970684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.971298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.973825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.974122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.974474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.974775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.978464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.979138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.981592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.982334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:33.985440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.985733 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.986043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.986139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.986372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.986460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.986651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.986732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.986927 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.987019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.987222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:33.987504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.002883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.003169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.003445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.003533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.003691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.003788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.003976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.004247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.010999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.011190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.012541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.012752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.012836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.013033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.015715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.015894 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.016014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.016185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.016319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.016481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.016608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.016772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.016887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.017052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.028144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.028407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.028490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.028693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.028776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.028971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.029091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.029249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.029340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.029522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.029603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.029820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.029905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.030092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.046837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.047632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.050311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.050624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.053198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.053541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.054776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.055153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.056251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.056541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.056639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.056828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.056910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.057198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.063179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.063688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.064769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.065322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.067708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.068082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.068453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.068844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.068931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.069006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.069006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.069135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.076396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.076684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.076962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.077177 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.077528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.077849 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.084194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=92260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.084477 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.084755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.084965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=92261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 16:39:34.085060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.085254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.085337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.085540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.085622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.085826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.085909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.086102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.086188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.086378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.086468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.086658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.086738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.086930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.087010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.087246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.087334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.087526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.087607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.087786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.087867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.088101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.088386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.088652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 16:39:34.121512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x564e4e64f9c0 pid: 92260 (default app) Aug 08 16:39:34.122752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x564e4e64f9c0 pid: 92261 (default app) Aug 08 16:39:34.124764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb07e-0b08-4c66-a0b5-5e11638acdb9 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:34.125074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb07e-0b08-4c66-a0b5-5e11638acdb9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:34.125345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb07e-0b08-4c66-a0b5-5e11638acdb9 None None] PATH_INFO: `/v3` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:34.129649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Mon Aug 8 16:39:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:42.638903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80775459-b68d-4cd8-a793-557a39ad562c None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:42.639432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80775459-b68d-4cd8-a793-557a39ad562c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:42.639432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80775459-b68d-4cd8-a793-557a39ad562c None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:42.644475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:42.649403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:42.649864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:42.649957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:42.719933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: INFO dbcounter [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] Registered counter for database keystone Aug 08 16:39:42.740668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG oslo_db.sqlalchemy.engines [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92260) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 16:39:42.746825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writer thread running {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 16:39:42.928769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG passlib.handlers.bcrypt [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92260) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 16:39:42.929095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG passlib.handlers.bcrypt [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92260) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 16:39:42.983465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:42.985082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:43.112622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 16:39:43.258409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80775459-b68d-4cd8-a793-557a39ad562c None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:43.327172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: INFO dbcounter [None req-80775459-b68d-4cd8-a793-557a39ad562c None None] Registered counter for database keystone Aug 08 16:39:43.347484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG oslo_db.sqlalchemy.engines [None req-80775459-b68d-4cd8-a793-557a39ad562c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92261) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 16:39:43.353776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writer thread running {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 16:39:43.472102 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80775459-b68d-4cd8-a793-557a39ad562c None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:43.476394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee662d3-a194-40ff-9952-b64e14f230b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:43.477010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee662d3-a194-40ff-9952-b64e14f230b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:43.477283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee662d3-a194-40ff-9952-b64e14f230b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:43.477576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee662d3-a194-40ff-9952-b64e14f230b1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:43.486736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee662d3-a194-40ff-9952-b64e14f230b1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:43.540955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3ee662d3-a194-40ff-9952-b64e14f230b1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 16:39:43.541219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee662d3-a194-40ff-9952-b64e14f230b1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:43.551675 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Aug 8 16:39:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 16:39:43.557870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:43.575671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172cfc1-a5ad-426d-848e-7bfea73d1515 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:43.580932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be761c30-d87b-416a-8ade-35407a5ab61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:43.582054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be761c30-d87b-416a-8ade-35407a5ab61e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:43.582341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be761c30-d87b-416a-8ade-35407a5ab61e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:43.582615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be761c30-d87b-416a-8ade-35407a5ab61e admin admin] PATH_INFO: `/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:43.584101 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be761c30-d87b-416a-8ade-35407a5ab61e admin admin] RBAC: Authorizing `identity:get_project(project_id=c0b6d65db9ee4a2cb85f51ab2f1fbc92)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:43.634307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: WARNING keystone.common.rbac_enforcer.enforcer [None req-be761c30-d87b-416a-8ade-35407a5ab61e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 16:39:43.634649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be761c30-d87b-416a-8ade-35407a5ab61e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:43.639387 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Mon Aug 8 16:39:43 2022] GET /identity/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:39:45.617676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e655b2-46fd-4efe-8f6c-42a89a443f62 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:45.618162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e655b2-46fd-4efe-8f6c-42a89a443f62 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:45.618253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e655b2-46fd-4efe-8f6c-42a89a443f62 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:45.622421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:45.628249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6e086-e444-4c5b-b55c-41849e11b7db None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:45.628643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6e086-e444-4c5b-b55c-41849e11b7db None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:45.628821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6e086-e444-4c5b-b55c-41849e11b7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:45.649697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-78d6e086-e444-4c5b-b55c-41849e11b7db None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:45.651249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-78d6e086-e444-4c5b-b55c-41849e11b7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:45.659170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 16:39:45.681261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e655b2-46fd-4efe-8f6c-42a89a443f62 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:45.693805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e655b2-46fd-4efe-8f6c-42a89a443f62 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:45.698009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d6a56-7787-4ff4-9507-0a1a81a730b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:45.698596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6a56-7787-4ff4-9507-0a1a81a730b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:45.698879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6a56-7787-4ff4-9507-0a1a81a730b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:45.699155 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6a56-7787-4ff4-9507-0a1a81a730b1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:45.707147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d6a56-7787-4ff4-9507-0a1a81a730b1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:45.708916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d6a56-7787-4ff4-9507-0a1a81a730b1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:45.717608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Aug 8 16:39:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 16:39:45.722963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6e086-e444-4c5b-b55c-41849e11b7db None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:45.731707 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6e086-e444-4c5b-b55c-41849e11b7db None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:45.736558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ad0fe-9457-4ab9-a4e1-2d2f35494747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:45.737311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ad0fe-9457-4ab9-a4e1-2d2f35494747 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:45.737588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ad0fe-9457-4ab9-a4e1-2d2f35494747 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:45.737892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ad0fe-9457-4ab9-a4e1-2d2f35494747 admin admin] PATH_INFO: `/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:45.739190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ad0fe-9457-4ab9-a4e1-2d2f35494747 admin admin] RBAC: Authorizing `identity:get_user(user_id=8502ad6c48f34f22831d3b9a1d5e8e5d)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:45.740980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ad0fe-9457-4ab9-a4e1-2d2f35494747 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:45.742502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Mon Aug 8 16:39:45 2022] GET /identity/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:47.787063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de849f-40c6-4150-a5be-2718c29bf286 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:47.787592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de849f-40c6-4150-a5be-2718c29bf286 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:47.787698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de849f-40c6-4150-a5be-2718c29bf286 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:47.788940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:47.794059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608a0e0-8eb2-4135-b643-942702851834 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:47.794472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608a0e0-8eb2-4135-b643-942702851834 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:47.794651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608a0e0-8eb2-4135-b643-942702851834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:47.801172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798fe98-7d16-40aa-b700-e25449e08d21 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:47.801453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798fe98-7d16-40aa-b700-e25449e08d21 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:47.801725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798fe98-7d16-40aa-b700-e25449e08d21 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:47.803037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:39:47.808214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e282-92f6-47fd-9925-8837e0ffd314 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:47.808495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e282-92f6-47fd-9925-8837e0ffd314 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:47.808767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e282-92f6-47fd-9925-8837e0ffd314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:47.813263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f608a0e0-8eb2-4135-b643-942702851834 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:47.814471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f608a0e0-8eb2-4135-b643-942702851834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:47.852227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG passlib.handlers.bcrypt [None req-4de7e282-92f6-47fd-9925-8837e0ffd314 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92261) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 16:39:47.852581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG passlib.handlers.bcrypt [None req-4de7e282-92f6-47fd-9925-8837e0ffd314 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92261) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 16:39:47.893213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:39:47.898513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4de7e282-92f6-47fd-9925-8837e0ffd314 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:47.900122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4de7e282-92f6-47fd-9925-8837e0ffd314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:47.915693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608a0e0-8eb2-4135-b643-942702851834 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:47.939750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:47.998091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:39:48.017933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608a0e0-8eb2-4135-b643-942702851834 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:48.019890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7e282-92f6-47fd-9925-8837e0ffd314 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:48.052860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6024d-ced4-4af6-a50b-89f744850c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:48.053487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6024d-ced4-4af6-a50b-89f744850c05 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:48.053760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6024d-ced4-4af6-a50b-89f744850c05 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:48.054061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6024d-ced4-4af6-a50b-89f744850c05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:48.065251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6024d-ced4-4af6-a50b-89f744850c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-95c6024d-ced4-4af6-a50b-89f744850c05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:39:48.073761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:39:48.074704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:39:48.075533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:39:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:39:48.082362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6024d-ced4-4af6-a50b-89f744850c05 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:48.118472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7e282-92f6-47fd-9925-8837e0ffd314 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:48.120637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:48.156060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b454f0-4995-4e89-a0f4-342a5b90b78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:48.156914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b454f0-4995-4e89-a0f4-342a5b90b78d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:48.157185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b454f0-4995-4e89-a0f4-342a5b90b78d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:48.157450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b454f0-4995-4e89-a0f4-342a5b90b78d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:48.163055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:48.182254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6024d-ced4-4af6-a50b-89f744850c05 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:48.218459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496a92b-0c34-4849-b3ab-e9a13eb12379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:48.219053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496a92b-0c34-4849-b3ab-e9a13eb12379 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:48.219332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496a92b-0c34-4849-b3ab-e9a13eb12379 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:48.219606 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496a92b-0c34-4849-b3ab-e9a13eb12379 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:48.220263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496a92b-0c34-4849-b3ab-e9a13eb12379 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:39:48.220563 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496a92b-0c34-4849-b3ab-e9a13eb12379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:48.221982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496a92b-0c34-4849-b3ab-e9a13eb12379 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:48.224862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b454f0-4995-4e89-a0f4-342a5b90b78d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:48.226430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b454f0-4995-4e89-a0f4-342a5b90b78d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:48.232902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:39:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:39:48.238419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496a92b-0c34-4849-b3ab-e9a13eb12379 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:48.329913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:39:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 16:39:48.335789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b454f0-4995-4e89-a0f4-342a5b90b78d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:48.338921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:48.437268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b454f0-4995-4e89-a0f4-342a5b90b78d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:48.445474 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496a92b-0c34-4849-b3ab-e9a13eb12379 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:48.470215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ce7d7-a4b5-42f3-b8f8-81a99cbb261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:48.471182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ce7d7-a4b5-42f3-b8f8-81a99cbb261a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:48.471355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ce7d7-a4b5-42f3-b8f8-81a99cbb261a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:48.471609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ce7d7-a4b5-42f3-b8f8-81a99cbb261a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:48.478293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56420c97-9fd3-4dff-b9e8-db0522d67468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:48.478912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56420c97-9fd3-4dff-b9e8-db0522d67468 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:48.479189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56420c97-9fd3-4dff-b9e8-db0522d67468 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:48.479456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56420c97-9fd3-4dff-b9e8-db0522d67468 None admin] PATH_INFO: `/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:48.480566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56420c97-9fd3-4dff-b9e8-db0522d67468 None admin] RBAC: Authorizing `identity:get_user(user_id=8502ad6c48f34f22831d3b9a1d5e8e5d)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:48.482028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56420c97-9fd3-4dff-b9e8-db0522d67468 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:48.484121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:39:48 2022] GET /identity/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:39:48.489428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56420c97-9fd3-4dff-b9e8-db0522d67468 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:48.491982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ce7d7-a4b5-42f3-b8f8-81a99cbb261a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:48.493453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ce7d7-a4b5-42f3-b8f8-81a99cbb261a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:48.503677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-741ce7d7-a4b5-42f3-b8f8-81a99cbb261a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:39:48.511390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:39:48.518582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:39:48.519435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:39:48.519435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:39:48.519435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:39:48.519435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:39:48.519435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ce7d7-a4b5-42f3-b8f8-81a99cbb261a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:48.573338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:48.587787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56420c97-9fd3-4dff-b9e8-db0522d67468 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:48.617123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ce7d7-a4b5-42f3-b8f8-81a99cbb261a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:48.624020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfe343-a981-477a-a091-205001aa6dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:48.624652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfe343-a981-477a-a091-205001aa6dca None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:48.624931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfe343-a981-477a-a091-205001aa6dca None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:48.625201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfe343-a981-477a-a091-205001aa6dca None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:48.626360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfe343-a981-477a-a091-205001aa6dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:48.627823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfe343-a981-477a-a091-205001aa6dca None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:48.629369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:39:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:39:48.635133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfe343-a981-477a-a091-205001aa6dca None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:48.650615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197ed30-07fa-4ebf-9055-8f7d83fcf781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:48.651236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197ed30-07fa-4ebf-9055-8f7d83fcf781 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:48.651516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197ed30-07fa-4ebf-9055-8f7d83fcf781 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:48.651790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197ed30-07fa-4ebf-9055-8f7d83fcf781 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:48.652433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197ed30-07fa-4ebf-9055-8f7d83fcf781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:39:48.652737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197ed30-07fa-4ebf-9055-8f7d83fcf781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:48.654151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197ed30-07fa-4ebf-9055-8f7d83fcf781 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:48.671478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:39:48.718449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:48.738265 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfe343-a981-477a-a091-205001aa6dca None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:48.774392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b7498-5bc6-4b75-873e-e5d873bdfc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:48.775016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b7498-5bc6-4b75-873e-e5d873bdfc25 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:48.775298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b7498-5bc6-4b75-873e-e5d873bdfc25 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:48.775572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b7498-5bc6-4b75-873e-e5d873bdfc25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:48.776314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b7498-5bc6-4b75-873e-e5d873bdfc25 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.domain.id=default, user.id=8502ad6c48f34f22831d3b9a1d5e8e5d) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:39:48.776620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b7498-5bc6-4b75-873e-e5d873bdfc25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:48.778133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b7498-5bc6-4b75-873e-e5d873bdfc25 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:48.786232 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Aug 8 16:39:48 2022] GET /identity/v3/role_assignments?user.id=8502ad6c48f34f22831d3b9a1d5e8e5d&scope.domain.id=default&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:39:51.070275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d661d7-0f1f-4909-9e8a-4a639b2031d2 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.070782 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d661d7-0f1f-4909-9e8a-4a639b2031d2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.070877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d661d7-0f1f-4909-9e8a-4a639b2031d2 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.072446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:51.077903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917a13f-40ff-4e0d-b16e-c932a4cccb59 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.078572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917a13f-40ff-4e0d-b16e-c932a4cccb59 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.078775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917a13f-40ff-4e0d-b16e-c932a4cccb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.101160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f917a13f-40ff-4e0d-b16e-c932a4cccb59 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:51.102411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f917a13f-40ff-4e0d-b16e-c932a4cccb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:51.192080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 12/23] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:39:51.215167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d661d7-0f1f-4909-9e8a-4a639b2031d2 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:51.272399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c429cda-bf32-465c-a94c-da9e11cbb7ba None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.272721 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c429cda-bf32-465c-a94c-da9e11cbb7ba None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.272949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c429cda-bf32-465c-a94c-da9e11cbb7ba None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.274313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:51.281884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e285a56-fd73-424f-b4f8-4a71f37c6756 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.282178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e285a56-fd73-424f-b4f8-4a71f37c6756 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.282437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e285a56-fd73-424f-b4f8-4a71f37c6756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.300679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1e285a56-fd73-424f-b4f8-4a71f37c6756 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:51.302216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1e285a56-fd73-424f-b4f8-4a71f37c6756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:51.375369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 14/25] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:39:51.389517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d661d7-0f1f-4909-9e8a-4a639b2031d2 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:51.401290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e285a56-fd73-424f-b4f8-4a71f37c6756 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:51.404580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:51.414684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:51.454540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb8214-041a-4cf4-ae80-bdea03253813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:51.455174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb8214-041a-4cf4-ae80-bdea03253813 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.455481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb8214-041a-4cf4-ae80-bdea03253813 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.455756 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb8214-041a-4cf4-ae80-bdea03253813 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.486774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb8214-041a-4cf4-ae80-bdea03253813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-10eb8214-041a-4cf4-ae80-bdea03253813 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:39:51.493143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:39:51.494008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:39:51.495199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 12/26] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:39:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:39:51.496022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64148d4d-77e9-48b6-b1bd-f72dc4200456 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.496022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64148d4d-77e9-48b6-b1bd-f72dc4200456 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.496022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64148d4d-77e9-48b6-b1bd-f72dc4200456 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.497210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 13/27] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:51.506776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64148d4d-77e9-48b6-b1bd-f72dc4200456 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:51.523431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e285a56-fd73-424f-b4f8-4a71f37c6756 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:51.590546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5aa0c-5b1f-4a79-bd32-0ef5a41d377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:51.591458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5aa0c-5b1f-4a79-bd32-0ef5a41d377e None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.594530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5aa0c-5b1f-4a79-bd32-0ef5a41d377e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.595038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5aa0c-5b1f-4a79-bd32-0ef5a41d377e None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.596045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5aa0c-5b1f-4a79-bd32-0ef5a41d377e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:51.597637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5aa0c-5b1f-4a79-bd32-0ef5a41d377e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:51.674613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64148d4d-77e9-48b6-b1bd-f72dc4200456 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:51.808615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 15/28] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Aug 8 16:39:51 2022] POST /identity/v3/roles => generated 219 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:39:51.810372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43f7c5-aa2a-4149-bef9-3e55ede110af None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.810800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43f7c5-aa2a-4149-bef9-3e55ede110af None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.811459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43f7c5-aa2a-4149-bef9-3e55ede110af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.833882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ae43f7c5-aa2a-4149-bef9-3e55ede110af None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:51.836209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ae43f7c5-aa2a-4149-bef9-3e55ede110af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:51.863088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da8be7-7847-412d-8bcc-ab99135a39e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:51.863802 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da8be7-7847-412d-8bcc-ab99135a39e7 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.863999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da8be7-7847-412d-8bcc-ab99135a39e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.864282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da8be7-7847-412d-8bcc-ab99135a39e7 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.864951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da8be7-7847-412d-8bcc-ab99135a39e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:39:51.865254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da8be7-7847-412d-8bcc-ab99135a39e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:51.866686 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da8be7-7847-412d-8bcc-ab99135a39e7 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:51.899760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 14/29] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:39:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:51.904053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fd071-c3ec-4a45-a4c7-6caa171bdd5f None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.906698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fd071-c3ec-4a45-a4c7-6caa171bdd5f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.907158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fd071-c3ec-4a45-a4c7-6caa171bdd5f None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.908775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 15/30] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:51.911168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fd071-c3ec-4a45-a4c7-6caa171bdd5f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:51.942299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:51.948897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 16/31] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:39:51.956052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a516210-7d6c-4c4f-b6f6-bbca1abae57d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:51.956555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a516210-7d6c-4c4f-b6f6-bbca1abae57d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:51.956967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a516210-7d6c-4c4f-b6f6-bbca1abae57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:51.987349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1a516210-7d6c-4c4f-b6f6-bbca1abae57d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:51.988883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1a516210-7d6c-4c4f-b6f6-bbca1abae57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:52.034175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:52.091246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 17/32] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:39:52.092893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a516210-7d6c-4c4f-b6f6-bbca1abae57d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:52.129506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fd071-c3ec-4a45-a4c7-6caa171bdd5f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:52.174682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01889d2-d5d0-4630-8bd5-568595bb11ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:52.174872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01889d2-d5d0-4630-8bd5-568595bb11ee None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.175120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01889d2-d5d0-4630-8bd5-568595bb11ee None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.175292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01889d2-d5d0-4630-8bd5-568595bb11ee None admin] PATH_INFO: `/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.177369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01889d2-d5d0-4630-8bd5-568595bb11ee None admin] RBAC: Authorizing `identity:get_user(user_id=8502ad6c48f34f22831d3b9a1d5e8e5d)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:52.179195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01889d2-d5d0-4630-8bd5-568595bb11ee None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:52.181546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 16/33] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:39:51 2022] GET /identity/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:52.182748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01889d2-d5d0-4630-8bd5-568595bb11ee None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:52.244077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a516210-7d6c-4c4f-b6f6-bbca1abae57d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:52.270216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:52.282353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9055a44-02bd-42aa-a1cc-a37997274f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:52.282972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9055a44-02bd-42aa-a1cc-a37997274f1e None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.283269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9055a44-02bd-42aa-a1cc-a37997274f1e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.283548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9055a44-02bd-42aa-a1cc-a37997274f1e None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.284230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9055a44-02bd-42aa-a1cc-a37997274f1e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:52.285717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9055a44-02bd-42aa-a1cc-a37997274f1e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:52.305576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01889d2-d5d0-4630-8bd5-568595bb11ee None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:52.313412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:52.341073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd5df9-8959-4743-8af9-529c513c34ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:52.341706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd5df9-8959-4743-8af9-529c513c34ec None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.342014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd5df9-8959-4743-8af9-529c513c34ec None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.342290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd5df9-8959-4743-8af9-529c513c34ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.352939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd5df9-8959-4743-8af9-529c513c34ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:52.354448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd5df9-8959-4743-8af9-529c513c34ec None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-2dfd5df9-8959-4743-8af9-529c513c34ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:39:52.368912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:39:52.369821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:39:52.370658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:39:52.370658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:39:52.370658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:39:52.370658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 17/34] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:39:52.371286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd5df9-8959-4743-8af9-529c513c34ec None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:52.379018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 18/35] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Aug 8 16:39:52 2022] POST /identity/v3/roles => generated 225 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:39:52.383134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416b209-5d4f-44e0-82d4-b541911a32e1 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.383134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416b209-5d4f-44e0-82d4-b541911a32e1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.383134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416b209-5d4f-44e0-82d4-b541911a32e1 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.384522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 19/36] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:52.387353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416b209-5d4f-44e0-82d4-b541911a32e1 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:52.501099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd5df9-8959-4743-8af9-529c513c34ec None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:52.539951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad2306-ce28-4d86-8cc1-bb08bc93f330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:52.540601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad2306-ce28-4d86-8cc1-bb08bc93f330 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.540923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad2306-ce28-4d86-8cc1-bb08bc93f330 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.541198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad2306-ce28-4d86-8cc1-bb08bc93f330 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.542422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad2306-ce28-4d86-8cc1-bb08bc93f330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:52.543959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad2306-ce28-4d86-8cc1-bb08bc93f330 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:52.546183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 18/37] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:39:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:52.552231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb8b44-55c0-4890-941a-556e2146ab4a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.552592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb8b44-55c0-4890-941a-556e2146ab4a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.552833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb8b44-55c0-4890-941a-556e2146ab4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.574070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-afeb8b44-55c0-4890-941a-556e2146ab4a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:52.575417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-afeb8b44-55c0-4890-941a-556e2146ab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:52.599075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:52.604135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416b209-5d4f-44e0-82d4-b541911a32e1 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:52.659175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578be55-18e9-493d-a53f-5a3c7de73b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:52.660251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578be55-18e9-493d-a53f-5a3c7de73b19 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.660807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578be55-18e9-493d-a53f-5a3c7de73b19 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.661247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578be55-18e9-493d-a53f-5a3c7de73b19 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.662269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578be55-18e9-493d-a53f-5a3c7de73b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:39:52.662769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578be55-18e9-493d-a53f-5a3c7de73b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:52.663685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 19/38] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:39:52.664652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578be55-18e9-493d-a53f-5a3c7de73b19 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:52.664849 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb8b44-55c0-4890-941a-556e2146ab4a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:52.691002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 20/39] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:52.691002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0c393-32fa-4c7e-8257-56cd90101a8b None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.691258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0c393-32fa-4c7e-8257-56cd90101a8b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.691534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0c393-32fa-4c7e-8257-56cd90101a8b None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.692792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 21/40] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:52.697421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0c393-32fa-4c7e-8257-56cd90101a8b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:52.745656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:52.779926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb8b44-55c0-4890-941a-556e2146ab4a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:52.799233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0c393-32fa-4c7e-8257-56cd90101a8b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:52.813449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44a95c-a8c5-449e-b54f-544bd62cb849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:52.814146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44a95c-a8c5-449e-b54f-544bd62cb849 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.814424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44a95c-a8c5-449e-b54f-544bd62cb849 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.814699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44a95c-a8c5-449e-b54f-544bd62cb849 None admin] PATH_INFO: `/v3/domains/default/users/8502ad6c48f34f22831d3b9a1d5e8e5d/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.816639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44a95c-a8c5-449e-b54f-544bd62cb849 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8502ad6c48f34f22831d3b9a1d5e8e5d, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:52.818099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44a95c-a8c5-449e-b54f-544bd62cb849 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:52.833481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8f33d-ae0e-456b-9130-b74bc5a1eb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:52.835311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8f33d-ae0e-456b-9130-b74bc5a1eb84 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:52.835311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8f33d-ae0e-456b-9130-b74bc5a1eb84 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:52.835311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8f33d-ae0e-456b-9130-b74bc5a1eb84 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:52.835311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8f33d-ae0e-456b-9130-b74bc5a1eb84 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:52.836640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8f33d-ae0e-456b-9130-b74bc5a1eb84 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:52.932970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 22/41] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Aug 8 16:39:52 2022] POST /identity/v3/roles => generated 223 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:39:52.934030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8f33d-ae0e-456b-9130-b74bc5a1eb84 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:53.019423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 20/42] 10.222.0.57 () {58 vars in 1361 bytes} [Mon Aug 8 16:39:52 2022] PUT /identity/v3/domains/default/users/8502ad6c48f34f22831d3b9a1d5e8e5d/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:39:53.019423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77442d-6bc0-437f-aca4-099c127088f5 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:53.019423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77442d-6bc0-437f-aca4-099c127088f5 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:53.019423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77442d-6bc0-437f-aca4-099c127088f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:53.034372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ed77442d-6bc0-437f-aca4-099c127088f5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:53.035917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ed77442d-6bc0-437f-aca4-099c127088f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:53.048628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:53.070943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8f33d-ae0e-456b-9130-b74bc5a1eb84 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:53.075325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:53.125745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 21/43] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:39:53.126372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87bee8-30f7-4695-b29e-fd5bda4434c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:53.127489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87bee8-30f7-4695-b29e-fd5bda4434c8 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:53.127769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87bee8-30f7-4695-b29e-fd5bda4434c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:53.128036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87bee8-30f7-4695-b29e-fd5bda4434c8 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:53.128705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87bee8-30f7-4695-b29e-fd5bda4434c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:53.130184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87bee8-30f7-4695-b29e-fd5bda4434c8 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:53.149453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77442d-6bc0-437f-aca4-099c127088f5 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:53.224745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 23/44] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:39:52 2022] POST /identity/v3/projects => generated 301 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:39:53.266670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77442d-6bc0-437f-aca4-099c127088f5 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:53.305348 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bb417-6c8d-44a0-8c16-61895b1645e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:53.305946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bb417-6c8d-44a0-8c16-61895b1645e6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:53.306236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bb417-6c8d-44a0-8c16-61895b1645e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:53.306508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bb417-6c8d-44a0-8c16-61895b1645e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:53.307704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bb417-6c8d-44a0-8c16-61895b1645e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:53.309461 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bb417-6c8d-44a0-8c16-61895b1645e6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:53.314294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 22/45] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:39:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:39:53.320572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87bee8-30f7-4695-b29e-fd5bda4434c8 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:53.432634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87bee8-30f7-4695-b29e-fd5bda4434c8 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:53.466891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401ffb6-f86e-40d9-8181-74d24c81af52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:53.468112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401ffb6-f86e-40d9-8181-74d24c81af52 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:53.468391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401ffb6-f86e-40d9-8181-74d24c81af52 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:53.468664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401ffb6-f86e-40d9-8181-74d24c81af52 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:53.469309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401ffb6-f86e-40d9-8181-74d24c81af52 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:53.470784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401ffb6-f86e-40d9-8181-74d24c81af52 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:53.568532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 24/46] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:39:53 2022] POST /identity/v3/projects => generated 312 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:39:55.027238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4da6a-dcf6-4518-b34d-3ccf0cbac5cb None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.027761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4da6a-dcf6-4518-b34d-3ccf0cbac5cb None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.027856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4da6a-dcf6-4518-b34d-3ccf0cbac5cb None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.029130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 23/47] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:55.035013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f7948-30d9-4e09-ac12-3fe7aba09d48 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.035428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f7948-30d9-4e09-ac12-3fe7aba09d48 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.035559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f7948-30d9-4e09-ac12-3fe7aba09d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.052920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-086f7948-30d9-4e09-ac12-3fe7aba09d48 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:55.054127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-086f7948-30d9-4e09-ac12-3fe7aba09d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:55.056045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:55.119533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 25/48] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:39:55.142864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4da6a-dcf6-4518-b34d-3ccf0cbac5cb None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:55.158775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:55.244404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4da6a-dcf6-4518-b34d-3ccf0cbac5cb None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:55.278294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df13e5-97dc-4102-85e2-cfa94e14819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:55.278852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df13e5-97dc-4102-85e2-cfa94e14819e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.279138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df13e5-97dc-4102-85e2-cfa94e14819e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.279406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df13e5-97dc-4102-85e2-cfa94e14819e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.289248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df13e5-97dc-4102-85e2-cfa94e14819e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-26df13e5-97dc-4102-85e2-cfa94e14819e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:39:55.293940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:39:55.294907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:39:55.295652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 24/49] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:39:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:39:55.301089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f7948-30d9-4e09-ac12-3fe7aba09d48 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:55.398642 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f7948-30d9-4e09-ac12-3fe7aba09d48 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:55.430865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4ccb0-060d-4c46-b97d-5bf92e069fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:55.431469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4ccb0-060d-4c46-b97d-5bf92e069fa6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.431746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4ccb0-060d-4c46-b97d-5bf92e069fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.432016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4ccb0-060d-4c46-b97d-5bf92e069fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.432676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4ccb0-060d-4c46-b97d-5bf92e069fa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:39:55.432980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4ccb0-060d-4c46-b97d-5bf92e069fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:55.434395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4ccb0-060d-4c46-b97d-5bf92e069fa6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:55.439899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:55.443227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 26/50] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:55.448760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df13e5-97dc-4102-85e2-cfa94e14819e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:55.523296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:55.547090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df13e5-97dc-4102-85e2-cfa94e14819e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:55.559099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ecbf-1d0a-4e6f-9a40-175ccf3dd586 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.559383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ecbf-1d0a-4e6f-9a40-175ccf3dd586 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.559653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ecbf-1d0a-4e6f-9a40-175ccf3dd586 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.561527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 27/51] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:39:55.566284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa4595-ae40-45a5-a370-0a34bae757b8 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.566559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa4595-ae40-45a5-a370-0a34bae757b8 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.566827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa4595-ae40-45a5-a370-0a34bae757b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.580971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e516f3-c68d-4860-a6f7-c28eb4ebe887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:55.581605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e516f3-c68d-4860-a6f7-c28eb4ebe887 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.581909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e516f3-c68d-4860-a6f7-c28eb4ebe887 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.582181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e516f3-c68d-4860-a6f7-c28eb4ebe887 None admin] PATH_INFO: `/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.583302 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e516f3-c68d-4860-a6f7-c28eb4ebe887 None admin] RBAC: Authorizing `identity:get_user(user_id=8502ad6c48f34f22831d3b9a1d5e8e5d)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:55.584448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2daa4595-ae40-45a5-a370-0a34bae757b8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:55.584789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e516f3-c68d-4860-a6f7-c28eb4ebe887 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:55.585629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2daa4595-ae40-45a5-a370-0a34bae757b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:55.586517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 25/52] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:39:55 2022] GET /identity/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:55.594703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e516f3-c68d-4860-a6f7-c28eb4ebe887 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:55.651713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 28/53] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:39:55.674510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa4595-ae40-45a5-a370-0a34bae757b8 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:55.692909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e516f3-c68d-4860-a6f7-c28eb4ebe887 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:55.729376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a4996-6c5c-4d67-a61a-90667f294cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:55.730040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a4996-6c5c-4d67-a61a-90667f294cba None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.730322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a4996-6c5c-4d67-a61a-90667f294cba None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.730602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a4996-6c5c-4d67-a61a-90667f294cba None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.731722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a4996-6c5c-4d67-a61a-90667f294cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:55.733795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a4996-6c5c-4d67-a61a-90667f294cba None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:55.735383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 26/54] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:39:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:55.741049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a4996-6c5c-4d67-a61a-90667f294cba None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:55.752548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:55.772515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa4595-ae40-45a5-a370-0a34bae757b8 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:55.801196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:55.806822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b0d38-363b-41be-98eb-4ca92e361f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:55.807451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b0d38-363b-41be-98eb-4ca92e361f1b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.807727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b0d38-363b-41be-98eb-4ca92e361f1b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.807995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b0d38-363b-41be-98eb-4ca92e361f1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.809153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b0d38-363b-41be-98eb-4ca92e361f1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:55.810638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b0d38-363b-41be-98eb-4ca92e361f1b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:55.812439 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 29/55] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:39:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:39:55.818030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b0d38-363b-41be-98eb-4ca92e361f1b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:55.838635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a4996-6c5c-4d67-a61a-90667f294cba None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:55.872807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229714-bd77-44ca-980b-454eaf91d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:55.873452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229714-bd77-44ca-980b-454eaf91d780 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.873732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229714-bd77-44ca-980b-454eaf91d780 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.874033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229714-bd77-44ca-980b-454eaf91d780 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.874757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229714-bd77-44ca-980b-454eaf91d780 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.domain.id=default, user.id=8502ad6c48f34f22831d3b9a1d5e8e5d) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:39:55.875064 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229714-bd77-44ca-980b-454eaf91d780 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:55.876507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229714-bd77-44ca-980b-454eaf91d780 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:55.884475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 27/56] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Aug 8 16:39:55 2022] GET /identity/v3/role_assignments?user.id=8502ad6c48f34f22831d3b9a1d5e8e5d&scope.domain.id=default&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:39:55.917043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b0d38-363b-41be-98eb-4ca92e361f1b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:55.953486 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4f4f3-df9e-4ea6-a7f9-cf918b7781fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:55.954137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f4f3-df9e-4ea6-a7f9-cf918b7781fc None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:55.954418 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f4f3-df9e-4ea6-a7f9-cf918b7781fc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:55.954692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f4f3-df9e-4ea6-a7f9-cf918b7781fc None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:55.955346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4f4f3-df9e-4ea6-a7f9-cf918b7781fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:55.956798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4f4f3-df9e-4ea6-a7f9-cf918b7781fc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:56.043014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 30/57] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Aug 8 16:39:55 2022] POST /identity/v3/projects => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:39:58.015299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89957417-5128-42ce-92c0-b9dd7fee5283 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:58.015826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89957417-5128-42ce-92c0-b9dd7fee5283 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:58.015912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89957417-5128-42ce-92c0-b9dd7fee5283 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:58.017195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 28/58] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:39:58.022439 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21297fd-da6c-4d80-809e-2f6e1b73f6dc None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:58.022855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21297fd-da6c-4d80-809e-2f6e1b73f6dc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:58.022983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21297fd-da6c-4d80-809e-2f6e1b73f6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:58.040357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b21297fd-da6c-4d80-809e-2f6e1b73f6dc None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:39:58.041520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b21297fd-da6c-4d80-809e-2f6e1b73f6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:39:58.085602 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:58.104746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 31/59] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:39:58.129089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89957417-5128-42ce-92c0-b9dd7fee5283 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:58.203361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:39:58.227893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89957417-5128-42ce-92c0-b9dd7fee5283 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:58.259792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7e4c8-23b7-478b-ab93-ff172044b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:58.260402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7e4c8-23b7-478b-ab93-ff172044b10d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:58.260681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7e4c8-23b7-478b-ab93-ff172044b10d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:58.260948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7e4c8-23b7-478b-ab93-ff172044b10d None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:58.262065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7e4c8-23b7-478b-ab93-ff172044b10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:58.263511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7e4c8-23b7-478b-ab93-ff172044b10d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:58.265741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 29/60] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:39:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:39:58.272543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21297fd-da6c-4d80-809e-2f6e1b73f6dc None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:39:58.377901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21297fd-da6c-4d80-809e-2f6e1b73f6dc None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:39:58.410090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51eaca-d636-43d3-abc2-07661e7a325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:39:58.410722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51eaca-d636-43d3-abc2-07661e7a325a None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:39:58.410999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51eaca-d636-43d3-abc2-07661e7a325a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:39:58.411266 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51eaca-d636-43d3-abc2-07661e7a325a None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:39:58.411894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51eaca-d636-43d3-abc2-07661e7a325a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:39:58.413322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51eaca-d636-43d3-abc2-07661e7a325a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:39:58.536978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 32/61] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:39:58 2022] POST /identity/v3/users => generated 275 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:01.698573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7812d-2771-4dc8-bb09-4e2cbda59c3e None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:01.699127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7812d-2771-4dc8-bb09-4e2cbda59c3e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:01.699127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7812d-2771-4dc8-bb09-4e2cbda59c3e None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:01.700450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 30/62] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:01.720233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d0cb9-258f-431d-b30a-369ae72725f4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:01.720740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d0cb9-258f-431d-b30a-369ae72725f4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:01.720821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d0cb9-258f-431d-b30a-369ae72725f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:01.746880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-972d0cb9-258f-431d-b30a-369ae72725f4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:01.748618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-972d0cb9-258f-431d-b30a-369ae72725f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:01.770718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d6eb3-7c77-4519-8760-55d5126823ea None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:01.771338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d6eb3-7c77-4519-8760-55d5126823ea None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:01.772248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d6eb3-7c77-4519-8760-55d5126823ea None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:01.773848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 31/63] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:40:01.780855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfb519-39e1-4086-b17d-a01c2b443f75 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:01.781727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfb519-39e1-4086-b17d-a01c2b443f75 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:01.782524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfb519-39e1-4086-b17d-a01c2b443f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:01.783796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:01.815224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6bcfb519-39e1-4086-b17d-a01c2b443f75 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:01.816806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6bcfb519-39e1-4086-b17d-a01c2b443f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:01.853884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 33/64] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:01.853884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114c6e7-3e13-4832-bfd4-1bd6bd627587 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:01.853884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114c6e7-3e13-4832-bfd4-1bd6bd627587 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:01.853884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114c6e7-3e13-4832-bfd4-1bd6bd627587 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:01.853884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 34/65] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:01.856863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12734a-460c-4b49-a68f-338f3d0cd23e None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:01.857261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12734a-460c-4b49-a68f-338f3d0cd23e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:01.858060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12734a-460c-4b49-a68f-338f3d0cd23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:01.879463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1a12734a-460c-4b49-a68f-338f3d0cd23e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:01.881410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1a12734a-460c-4b49-a68f-338f3d0cd23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:01.907995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 32/66] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:01.908728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfb519-39e1-4086-b17d-a01c2b443f75 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:01.979181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 35/67] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:01.986835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12734a-460c-4b49-a68f-338f3d0cd23e None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:01.988423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:02.014793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfb519-39e1-4086-b17d-a01c2b443f75 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.055183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf116be-a2e2-47d1-b5b6-f473217dac46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.055829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf116be-a2e2-47d1-b5b6-f473217dac46 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.056109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf116be-a2e2-47d1-b5b6-f473217dac46 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.056405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf116be-a2e2-47d1-b5b6-f473217dac46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.071701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf116be-a2e2-47d1-b5b6-f473217dac46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-ecf116be-a2e2-47d1-b5b6-f473217dac46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:02.079698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:02.080572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:02.081341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:02.083173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 33/68] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:02.084844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf116be-a2e2-47d1-b5b6-f473217dac46 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.121168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:02.128673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12734a-460c-4b49-a68f-338f3d0cd23e None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.168519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b5117-44d2-4975-a24a-6cfd57cd3abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.169652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b5117-44d2-4975-a24a-6cfd57cd3abe None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.169652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b5117-44d2-4975-a24a-6cfd57cd3abe None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.169869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b5117-44d2-4975-a24a-6cfd57cd3abe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.180202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b5117-44d2-4975-a24a-6cfd57cd3abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-606b5117-44d2-4975-a24a-6cfd57cd3abe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:02.184906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:02.185774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:02.186641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 36/69] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:40:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:02.187249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b5117-44d2-4975-a24a-6cfd57cd3abe None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.208649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf116be-a2e2-47d1-b5b6-f473217dac46 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.240320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:02.245938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f90f0-70b7-4ba2-a3c7-999b07be0fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.246644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f90f0-70b7-4ba2-a3c7-999b07be0fb9 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.246932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f90f0-70b7-4ba2-a3c7-999b07be0fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.247208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f90f0-70b7-4ba2-a3c7-999b07be0fb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.258045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f90f0-70b7-4ba2-a3c7-999b07be0fb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-c07f90f0-70b7-4ba2-a3c7-999b07be0fb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:02.262699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:02.264140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 34/70] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:02.269920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894768c-3121-4539-bb9b-b3620081c63f None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.270603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894768c-3121-4539-bb9b-b3620081c63f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.270603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894768c-3121-4539-bb9b-b3620081c63f None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.270603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 35/71] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:02.270603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894768c-3121-4539-bb9b-b3620081c63f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.297097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b5117-44d2-4975-a24a-6cfd57cd3abe None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.331864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b92b8-cde6-438e-a1d0-bfb43d168d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.332475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b92b8-cde6-438e-a1d0-bfb43d168d61 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.332764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b92b8-cde6-438e-a1d0-bfb43d168d61 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.333039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b92b8-cde6-438e-a1d0-bfb43d168d61 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.333705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b92b8-cde6-438e-a1d0-bfb43d168d61 None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:02.334042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b92b8-cde6-438e-a1d0-bfb43d168d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:02.335447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b92b8-cde6-438e-a1d0-bfb43d168d61 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:02.343860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 37/72] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:02.348727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d801e-dd3d-48f1-a7bc-3fbd6771d984 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.348727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d801e-dd3d-48f1-a7bc-3fbd6771d984 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.348727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d801e-dd3d-48f1-a7bc-3fbd6771d984 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.348727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 38/73] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:02.348727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d801e-dd3d-48f1-a7bc-3fbd6771d984 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.366437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:02.379214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894768c-3121-4539-bb9b-b3620081c63f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.408331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5541c3-6132-4ed8-be7f-355322f62fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.408939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5541c3-6132-4ed8-be7f-355322f62fc3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.409212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5541c3-6132-4ed8-be7f-355322f62fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.409483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5541c3-6132-4ed8-be7f-355322f62fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.410176 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5541c3-6132-4ed8-be7f-355322f62fc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:02.410477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5541c3-6132-4ed8-be7f-355322f62fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:02.411855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5541c3-6132-4ed8-be7f-355322f62fc3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:02.419699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 36/74] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:02.420818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e7c04-47e2-409e-a381-e7e1a28acf86 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.421099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e7c04-47e2-409e-a381-e7e1a28acf86 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.421367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e7c04-47e2-409e-a381-e7e1a28acf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.438530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-fd3e7c04-47e2-409e-a381-e7e1a28acf86 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:02.439705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fd3e7c04-47e2-409e-a381-e7e1a28acf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:02.449467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d801e-dd3d-48f1-a7bc-3fbd6771d984 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.482999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cddb2-3d45-47bf-9ecd-5b8a169faab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.483601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cddb2-3d45-47bf-9ecd-5b8a169faab3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.483881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cddb2-3d45-47bf-9ecd-5b8a169faab3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.484152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cddb2-3d45-47bf-9ecd-5b8a169faab3 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.484815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cddb2-3d45-47bf-9ecd-5b8a169faab3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:02.485116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cddb2-3d45-47bf-9ecd-5b8a169faab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:02.486537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cddb2-3d45-47bf-9ecd-5b8a169faab3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:02.487927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:02.495058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 39/75] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:02.495839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cddb2-3d45-47bf-9ecd-5b8a169faab3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.510559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 37/76] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:02.511829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35c056-b522-4281-874e-240f06cc5ca4 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.512144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35c056-b522-4281-874e-240f06cc5ca4 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.512415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35c056-b522-4281-874e-240f06cc5ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.530197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-db35c056-b522-4281-874e-240f06cc5ca4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:02.531356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-db35c056-b522-4281-874e-240f06cc5ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:02.589341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:02.597685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 38/77] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:02.602355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35c056-b522-4281-874e-240f06cc5ca4 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.610121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cddb2-3d45-47bf-9ecd-5b8a169faab3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.645218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe7515-7bdb-464b-9016-2ff48740793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.645775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe7515-7bdb-464b-9016-2ff48740793a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.646082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe7515-7bdb-464b-9016-2ff48740793a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.646353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe7515-7bdb-464b-9016-2ff48740793a None admin] PATH_INFO: `/v3/users/3b9f97788d554a69b9c325c4e2da6890` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.659063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe7515-7bdb-464b-9016-2ff48740793a None admin] RBAC: Authorizing `identity:get_user(user_id=3b9f97788d554a69b9c325c4e2da6890)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:02.660524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe7515-7bdb-464b-9016-2ff48740793a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:02.662446 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 40/78] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/users/3b9f97788d554a69b9c325c4e2da6890 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:02.663212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe7515-7bdb-464b-9016-2ff48740793a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.699525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35c056-b522-4281-874e-240f06cc5ca4 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.713944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:02.732854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73406-f33c-499f-a4bd-423d6cb076e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.733767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73406-f33c-499f-a4bd-423d6cb076e5 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.733911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73406-f33c-499f-a4bd-423d6cb076e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.734135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73406-f33c-499f-a4bd-423d6cb076e5 None admin] PATH_INFO: `/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.735290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b73406-f33c-499f-a4bd-423d6cb076e5 None admin] RBAC: Authorizing `identity:get_user(user_id=8502ad6c48f34f22831d3b9a1d5e8e5d)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:02.736738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b73406-f33c-499f-a4bd-423d6cb076e5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:02.738354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 39/79] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:02.739116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73406-f33c-499f-a4bd-423d6cb076e5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.761261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe7515-7bdb-464b-9016-2ff48740793a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.794606 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b67cc4-db80-4e08-8499-4c3b23f9971c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.795237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b67cc4-db80-4e08-8499-4c3b23f9971c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.795519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b67cc4-db80-4e08-8499-4c3b23f9971c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.795792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b67cc4-db80-4e08-8499-4c3b23f9971c None admin] PATH_INFO: `/v3/users/3b9f97788d554a69b9c325c4e2da6890` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.796901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b67cc4-db80-4e08-8499-4c3b23f9971c None admin] RBAC: Authorizing `identity:get_user(user_id=3b9f97788d554a69b9c325c4e2da6890)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:02.798385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b67cc4-db80-4e08-8499-4c3b23f9971c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:02.799948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 41/80] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/users/3b9f97788d554a69b9c325c4e2da6890 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:02.804503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b67cc4-db80-4e08-8499-4c3b23f9971c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.809344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:02.838723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73406-f33c-499f-a4bd-423d6cb076e5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.870942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d696d84-e46d-49cf-b6a8-c620c014cc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.871571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d696d84-e46d-49cf-b6a8-c620c014cc22 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.871851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d696d84-e46d-49cf-b6a8-c620c014cc22 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.872122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d696d84-e46d-49cf-b6a8-c620c014cc22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.873830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d696d84-e46d-49cf-b6a8-c620c014cc22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:02.875273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d696d84-e46d-49cf-b6a8-c620c014cc22 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:02.876840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 40/81] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:02.877707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d696d84-e46d-49cf-b6a8-c620c014cc22 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.900847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b67cc4-db80-4e08-8499-4c3b23f9971c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:02.926344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:02.934151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e611021-20aa-4f27-821d-eb76cec28d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:02.936585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e611021-20aa-4f27-821d-eb76cec28d0d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:02.936585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e611021-20aa-4f27-821d-eb76cec28d0d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:02.936585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e611021-20aa-4f27-821d-eb76cec28d0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:02.945729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e611021-20aa-4f27-821d-eb76cec28d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-0e611021-20aa-4f27-821d-eb76cec28d0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:02.950841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:02.951754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:02.952751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 42/82] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:02.953381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e611021-20aa-4f27-821d-eb76cec28d0d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:02.977250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d696d84-e46d-49cf-b6a8-c620c014cc22 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:03.014693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6904-c271-493f-99c0-53082eaefb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:03.015546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6904-c271-493f-99c0-53082eaefb25 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:03.015831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6904-c271-493f-99c0-53082eaefb25 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:03.016105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6904-c271-493f-99c0-53082eaefb25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:03.020289 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:03.056797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e611021-20aa-4f27-821d-eb76cec28d0d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:03.083897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6904-c271-493f-99c0-53082eaefb25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:03.085391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6904-c271-493f-99c0-53082eaefb25 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:03.090087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ea8a-f931-428f-8e5b-caf1aa3ba7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:03.090691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ea8a-f931-428f-8e5b-caf1aa3ba7ec None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:03.090971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ea8a-f931-428f-8e5b-caf1aa3ba7ec None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:03.091244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ea8a-f931-428f-8e5b-caf1aa3ba7ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:03.133054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:03.159168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ea8a-f931-428f-8e5b-caf1aa3ba7ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:03.160563 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ea8a-f931-428f-8e5b-caf1aa3ba7ec None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:03.182062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 41/83] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:03.182902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6904-c271-493f-99c0-53082eaefb25 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:03.227931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:03.262532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 43/84] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:03.263362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ea8a-f931-428f-8e5b-caf1aa3ba7ec None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:03.280939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6904-c271-493f-99c0-53082eaefb25 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:03.313983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ff313-6c5d-4ea7-977d-dc71156ad332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:03.314594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ff313-6c5d-4ea7-977d-dc71156ad332 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:03.314877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ff313-6c5d-4ea7-977d-dc71156ad332 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:03.315152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ff313-6c5d-4ea7-977d-dc71156ad332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:03.334344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:03.360406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ea8a-f931-428f-8e5b-caf1aa3ba7ec None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:03.383032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ff313-6c5d-4ea7-977d-dc71156ad332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:03.384505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ff313-6c5d-4ea7-977d-dc71156ad332 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:03.394210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb6228-2df4-42bc-b18b-2318748bcfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:03.394816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6228-2df4-42bc-b18b-2318748bcfcb None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:03.395098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6228-2df4-42bc-b18b-2318748bcfcb None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:03.395374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6228-2df4-42bc-b18b-2318748bcfcb None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:03.396011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb6228-2df4-42bc-b18b-2318748bcfcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:03.397438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb6228-2df4-42bc-b18b-2318748bcfcb None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:03.479498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 44/85] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:03 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:03.480078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 42/86] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:03.480813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb6228-2df4-42bc-b18b-2318748bcfcb None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:03.481003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ff313-6c5d-4ea7-977d-dc71156ad332 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:03.510691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:03.542525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:03.600944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ff313-6c5d-4ea7-977d-dc71156ad332 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:03.605904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb6228-2df4-42bc-b18b-2318748bcfcb None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:03.640436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec78bb-dc2a-45f2-88c2-889131951195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:03.641121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec78bb-dc2a-45f2-88c2-889131951195 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:03.642055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec78bb-dc2a-45f2-88c2-889131951195 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:03.642055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec78bb-dc2a-45f2-88c2-889131951195 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:03.642956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec78bb-dc2a-45f2-88c2-889131951195 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:03.645762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec78bb-dc2a-45f2-88c2-889131951195 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:03.647140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 43/87] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:03 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:03.647140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec78bb-dc2a-45f2-88c2-889131951195 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:03.653956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e210407-2898-4a0e-ae2a-fbc19970ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:03.655222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e210407-2898-4a0e-ae2a-fbc19970ce0a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:03.655476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e210407-2898-4a0e-ae2a-fbc19970ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:03.655752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e210407-2898-4a0e-ae2a-fbc19970ce0a None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:03.656432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e210407-2898-4a0e-ae2a-fbc19970ce0a None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:03.656736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e210407-2898-4a0e-ae2a-fbc19970ce0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:03.658194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e210407-2898-4a0e-ae2a-fbc19970ce0a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:03.668246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 45/88] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:03.674187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e210407-2898-4a0e-ae2a-fbc19970ce0a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:03.761887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec78bb-dc2a-45f2-88c2-889131951195 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:03.780937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:03.794970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:03.807885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c6741-1b50-4a1a-84a6-37987358089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:03.808840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c6741-1b50-4a1a-84a6-37987358089b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:03.809506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c6741-1b50-4a1a-84a6-37987358089b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:03.810349 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c6741-1b50-4a1a-84a6-37987358089b None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:03.812004 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c6741-1b50-4a1a-84a6-37987358089b None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:03.812963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e210407-2898-4a0e-ae2a-fbc19970ce0a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:03.813898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c6741-1b50-4a1a-84a6-37987358089b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:03.817101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 44/89] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:03 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:03.818240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c6741-1b50-4a1a-84a6-37987358089b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:03.860747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707edaef-4acc-4f43-b916-30a0b2899df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:03.861682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707edaef-4acc-4f43-b916-30a0b2899df7 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:03.862235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707edaef-4acc-4f43-b916-30a0b2899df7 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:03.862736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707edaef-4acc-4f43-b916-30a0b2899df7 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:03.864085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707edaef-4acc-4f43-b916-30a0b2899df7 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:03.866216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707edaef-4acc-4f43-b916-30a0b2899df7 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:03.870157 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 46/90] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:03 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:03.871150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707edaef-4acc-4f43-b916-30a0b2899df7 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:03.935691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c6741-1b50-4a1a-84a6-37987358089b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:03.974946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e3fd3-8e4b-4dec-9fd7-802e2e7e56b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:03.975557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3fd3-8e4b-4dec-9fd7-802e2e7e56b3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:03.975886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3fd3-8e4b-4dec-9fd7-802e2e7e56b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:03.976211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3fd3-8e4b-4dec-9fd7-802e2e7e56b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:03.976940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e3fd3-8e4b-4dec-9fd7-802e2e7e56b3 None admin] RBAC: Adding query filter params (role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0, user.id=3b9f97788d554a69b9c325c4e2da6890) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:03.977251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e3fd3-8e4b-4dec-9fd7-802e2e7e56b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:03.978788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e3fd3-8e4b-4dec-9fd7-802e2e7e56b3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:03.984532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 45/91] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:03 2022] GET /identity/v3/role_assignments?user.id=3b9f97788d554a69b9c325c4e2da6890&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:03.985342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e3fd3-8e4b-4dec-9fd7-802e2e7e56b3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:04.003120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707edaef-4acc-4f43-b916-30a0b2899df7 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:04.048042 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:04.050132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a796b-3334-4e19-892b-63b3f3aa7b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:04.050986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a796b-3334-4e19-892b-63b3f3aa7b3a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:04.051532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a796b-3334-4e19-892b-63b3f3aa7b3a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:04.052007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a796b-3334-4e19-892b-63b3f3aa7b3a None admin] PATH_INFO: `/v3/users/3b9f97788d554a69b9c325c4e2da6890` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:04.053730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a796b-3334-4e19-892b-63b3f3aa7b3a None admin] RBAC: Authorizing `identity:get_user(user_id=3b9f97788d554a69b9c325c4e2da6890)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:04.056303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a796b-3334-4e19-892b-63b3f3aa7b3a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:04.058882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 47/92] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:03 2022] GET /identity/v3/users/3b9f97788d554a69b9c325c4e2da6890 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:04.060598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a796b-3334-4e19-892b-63b3f3aa7b3a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:04.087154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:04.115360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e3fd3-8e4b-4dec-9fd7-802e2e7e56b3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:04.190512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10dd29-0fff-4c19-90ed-b95eb626c442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:04.191138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10dd29-0fff-4c19-90ed-b95eb626c442 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:04.191425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10dd29-0fff-4c19-90ed-b95eb626c442 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:04.191699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10dd29-0fff-4c19-90ed-b95eb626c442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:04.192439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10dd29-0fff-4c19-90ed-b95eb626c442 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0, user.id=8502ad6c48f34f22831d3b9a1d5e8e5d) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:04.192749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10dd29-0fff-4c19-90ed-b95eb626c442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:04.194244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10dd29-0fff-4c19-90ed-b95eb626c442 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:04.206759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 46/93] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:03 2022] GET /identity/v3/role_assignments?user.id=8502ad6c48f34f22831d3b9a1d5e8e5d&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:04.207610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10dd29-0fff-4c19-90ed-b95eb626c442 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:04.217837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a796b-3334-4e19-892b-63b3f3aa7b3a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:04.257980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857ce2d-3736-49cc-ac75-5881a73e1c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:04.258603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857ce2d-3736-49cc-ac75-5881a73e1c14 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:04.258888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857ce2d-3736-49cc-ac75-5881a73e1c14 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:04.259166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857ce2d-3736-49cc-ac75-5881a73e1c14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:04.259908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857ce2d-3736-49cc-ac75-5881a73e1c14 None admin] RBAC: Adding query filter params (role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0, user.id=3b9f97788d554a69b9c325c4e2da6890) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:04.260220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857ce2d-3736-49cc-ac75-5881a73e1c14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:04.261713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857ce2d-3736-49cc-ac75-5881a73e1c14 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:04.270625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 48/94] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:04 2022] GET /identity/v3/role_assignments?user.id=3b9f97788d554a69b9c325c4e2da6890&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:04.423909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10dd29-0fff-4c19-90ed-b95eb626c442 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:04.444565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:04.467013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c83dc-de43-485d-9ade-c0a410392b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:04.467636 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c83dc-de43-485d-9ade-c0a410392b88 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:04.467919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c83dc-de43-485d-9ade-c0a410392b88 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:04.468194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c83dc-de43-485d-9ade-c0a410392b88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:04.540924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c83dc-de43-485d-9ade-c0a410392b88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:04.542579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c83dc-de43-485d-9ade-c0a410392b88 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:04.646233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 47/95] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:04.652332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857ce2d-3736-49cc-ac75-5881a73e1c14 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:04.774511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:04.790935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857ce2d-3736-49cc-ac75-5881a73e1c14 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:04.832372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7084-69f5-47cd-b29a-0f0ec1f67240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:04.832985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7084-69f5-47cd-b29a-0f0ec1f67240 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:04.833260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7084-69f5-47cd-b29a-0f0ec1f67240 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:04.833532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7084-69f5-47cd-b29a-0f0ec1f67240 None admin] PATH_INFO: `/v3/projects/ccf3799e8d9e4dcb8a61bea731cb301d` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:04.834795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7084-69f5-47cd-b29a-0f0ec1f67240 None admin] RBAC: Authorizing `identity:get_project(project_id=ccf3799e8d9e4dcb8a61bea731cb301d)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:04.836229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7084-69f5-47cd-b29a-0f0ec1f67240 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:04.838098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 49/96] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:04 2022] GET /identity/v3/projects/ccf3799e8d9e4dcb8a61bea731cb301d => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:04.844672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c83dc-de43-485d-9ade-c0a410392b88 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:04.864322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:04.963698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c83dc-de43-485d-9ade-c0a410392b88 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:04.999945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e245d-e10f-4d1f-aa39-e29f64ea71fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:05.000564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e245d-e10f-4d1f-aa39-e29f64ea71fb None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:05.000851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e245d-e10f-4d1f-aa39-e29f64ea71fb None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:05.001133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e245d-e10f-4d1f-aa39-e29f64ea71fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:05.001892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e245d-e10f-4d1f-aa39-e29f64ea71fb None admin] RBAC: Adding query filter params (role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=ccf3799e8d9e4dcb8a61bea731cb301d, user.id=3b9f97788d554a69b9c325c4e2da6890) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:05.002211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e245d-e10f-4d1f-aa39-e29f64ea71fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:05.003663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e245d-e10f-4d1f-aa39-e29f64ea71fb None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:05.009629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 48/97] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:04 2022] GET /identity/v3/role_assignments?user.id=3b9f97788d554a69b9c325c4e2da6890&scope.project.id=ccf3799e8d9e4dcb8a61bea731cb301d&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:06.352498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241b8b3-0187-4eb0-bda3-001b2138ea30 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:06.353041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241b8b3-0187-4eb0-bda3-001b2138ea30 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:06.353359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241b8b3-0187-4eb0-bda3-001b2138ea30 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:06.354730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 50/98] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:06.360435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c249-882d-46bc-8510-fee2292d2a7f None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:06.360894 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c249-882d-46bc-8510-fee2292d2a7f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:06.361071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c249-882d-46bc-8510-fee2292d2a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:06.384137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3222c249-882d-46bc-8510-fee2292d2a7f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:06.385342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3222c249-882d-46bc-8510-fee2292d2a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:06.471414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 49/99] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:06.504173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241b8b3-0187-4eb0-bda3-001b2138ea30 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:06.619529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241b8b3-0187-4eb0-bda3-001b2138ea30 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:06.663470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8c755-ae28-401b-80cd-6db26de934ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:06.664108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8c755-ae28-401b-80cd-6db26de934ac None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:06.664394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8c755-ae28-401b-80cd-6db26de934ac None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:06.664673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8c755-ae28-401b-80cd-6db26de934ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:06.665813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8c755-ae28-401b-80cd-6db26de934ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:06.667368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8c755-ae28-401b-80cd-6db26de934ac None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:06.668952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 51/100] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:06.674969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222c249-882d-46bc-8510-fee2292d2a7f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:06.690149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:06.760214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d272078-1a45-49cf-a9c4-c5bd3500a4ab None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:06.760553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d272078-1a45-49cf-a9c4-c5bd3500a4ab None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:06.760774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d272078-1a45-49cf-a9c4-c5bd3500a4ab None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:06.762151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 52/101] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:40:06.779526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168feed2-0cb2-4202-8e07-44c04397e6ee None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:06.779820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168feed2-0cb2-4202-8e07-44c04397e6ee None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:06.780080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168feed2-0cb2-4202-8e07-44c04397e6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:06.809686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-168feed2-0cb2-4202-8e07-44c04397e6ee None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:06.811267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-168feed2-0cb2-4202-8e07-44c04397e6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:06.817808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222c249-882d-46bc-8510-fee2292d2a7f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:06.861582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:06.865020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12713f20-cfbb-4497-807a-96f70fa4785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:06.865811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12713f20-cfbb-4497-807a-96f70fa4785f None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:06.866103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12713f20-cfbb-4497-807a-96f70fa4785f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:06.866382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12713f20-cfbb-4497-807a-96f70fa4785f None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:06.867434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12713f20-cfbb-4497-807a-96f70fa4785f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:06.869074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12713f20-cfbb-4497-807a-96f70fa4785f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:06.974641 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 50/102] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:40:06 2022] POST /identity/v3/projects => generated 302 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:06.976177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 53/103] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:07.010574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12713f20-cfbb-4497-807a-96f70fa4785f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:07.096441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:07.108246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37296198-b997-4820-89e1-a5a0c4006451 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.108895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37296198-b997-4820-89e1-a5a0c4006451 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.109453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37296198-b997-4820-89e1-a5a0c4006451 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.111131 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 54/104] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:07.120804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829f375-9e13-4cf6-9b0e-99456b036e83 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.122228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829f375-9e13-4cf6-9b0e-99456b036e83 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.122611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829f375-9e13-4cf6-9b0e-99456b036e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.155758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-7829f375-9e13-4cf6-9b0e-99456b036e83 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:07.156963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7829f375-9e13-4cf6-9b0e-99456b036e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:07.182212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12713f20-cfbb-4497-807a-96f70fa4785f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:07.264211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfcbca-e286-4849-8153-414298a939d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:07.265582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfcbca-e286-4849-8153-414298a939d0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.265582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfcbca-e286-4849-8153-414298a939d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.265582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfcbca-e286-4849-8153-414298a939d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.278451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedfcbca-e286-4849-8153-414298a939d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-aedfcbca-e286-4849-8153-414298a939d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:07.284431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:07.285983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:07.286737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 51/105] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:07.293684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfcbca-e286-4849-8153-414298a939d0 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:07.343941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 55/106] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:07.345591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c1485-d15e-42a3-93b8-058e9c653788 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.345918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c1485-d15e-42a3-93b8-058e9c653788 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.346199 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c1485-d15e-42a3-93b8-058e9c653788 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.347637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 56/107] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:07.352773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f317700-0d63-4088-929d-f8355de7bddb None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.353412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f317700-0d63-4088-929d-f8355de7bddb None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.353412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f317700-0d63-4088-929d-f8355de7bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.376767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0f317700-0d63-4088-929d-f8355de7bddb None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:07.377989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0f317700-0d63-4088-929d-f8355de7bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:07.585896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfcbca-e286-4849-8153-414298a939d0 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:07.597118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 57/108] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:07.606782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f317700-0d63-4088-929d-f8355de7bddb None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:07.607129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:07.624322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a195f-7f9b-42aa-a4c8-923cc34c52b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:07.625019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a195f-7f9b-42aa-a4c8-923cc34c52b2 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.625309 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a195f-7f9b-42aa-a4c8-923cc34c52b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.634412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a195f-7f9b-42aa-a4c8-923cc34c52b2 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.634412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a195f-7f9b-42aa-a4c8-923cc34c52b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:07.634412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a195f-7f9b-42aa-a4c8-923cc34c52b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:07.634412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a195f-7f9b-42aa-a4c8-923cc34c52b2 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:07.634671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:07.638847 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 52/109] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:07.640214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be52cb-41be-4c63-8fd4-f2616105a9ec None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.640585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be52cb-41be-4c63-8fd4-f2616105a9ec None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.640933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be52cb-41be-4c63-8fd4-f2616105a9ec None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.642380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 53/110] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:07.643425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be52cb-41be-4c63-8fd4-f2616105a9ec None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:07.753669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f317700-0d63-4088-929d-f8355de7bddb None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:07.789820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd16b7b-8394-46f9-a53c-c8ad0fac2f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:07.789820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd16b7b-8394-46f9-a53c-c8ad0fac2f75 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.789820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd16b7b-8394-46f9-a53c-c8ad0fac2f75 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.789820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd16b7b-8394-46f9-a53c-c8ad0fac2f75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.799471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be52cb-41be-4c63-8fd4-f2616105a9ec None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:07.800749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd16b7b-8394-46f9-a53c-c8ad0fac2f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-8dd16b7b-8394-46f9-a53c-c8ad0fac2f75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:07.805497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:07.806365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:07.807125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 58/111] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:07.808527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd8617-6e21-4b45-9afb-6234f298fd65 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.808527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd8617-6e21-4b45-9afb-6234f298fd65 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.808527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd8617-6e21-4b45-9afb-6234f298fd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.827462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-37fd8617-6e21-4b45-9afb-6234f298fd65 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:07.828918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-37fd8617-6e21-4b45-9afb-6234f298fd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:07.842643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf8406-a1c2-4d49-9f5b-1366b15e8d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:07.843181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf8406-a1c2-4d49-9f5b-1366b15e8d81 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.843460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf8406-a1c2-4d49-9f5b-1366b15e8d81 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.843731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf8406-a1c2-4d49-9f5b-1366b15e8d81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.854390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf8406-a1c2-4d49-9f5b-1366b15e8d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-5fbf8406-a1c2-4d49-9f5b-1366b15e8d81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:07.859106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:07.859936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:07.860678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 54/112] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:40:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:07.861336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf8406-a1c2-4d49-9f5b-1366b15e8d81 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:07.908635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:07.920693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 59/113] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:07.926325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:07.926430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cd9ba-c6fc-4309-baa6-05c4fffd5012 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:07.926430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cd9ba-c6fc-4309-baa6-05c4fffd5012 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:07.926430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cd9ba-c6fc-4309-baa6-05c4fffd5012 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:07.938429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 60/114] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:07 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:07.938429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cd9ba-c6fc-4309-baa6-05c4fffd5012 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:07.979780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf8406-a1c2-4d49-9f5b-1366b15e8d81 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.019370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ca9dd-3653-41a1-bfb7-7b42e3d3f2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.020763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ca9dd-3653-41a1-bfb7-7b42e3d3f2f5 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.021049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ca9dd-3653-41a1-bfb7-7b42e3d3f2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.021322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ca9dd-3653-41a1-bfb7-7b42e3d3f2f5 None admin] PATH_INFO: `/v3/users/3b9f97788d554a69b9c325c4e2da6890` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.023408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ca9dd-3653-41a1-bfb7-7b42e3d3f2f5 None admin] RBAC: Authorizing `identity:get_user(user_id=3b9f97788d554a69b9c325c4e2da6890)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:08.025562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ca9dd-3653-41a1-bfb7-7b42e3d3f2f5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:08.033257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 55/115] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:07 2022] GET /identity/v3/users/3b9f97788d554a69b9c325c4e2da6890 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:08.033257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ca9dd-3653-41a1-bfb7-7b42e3d3f2f5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.059173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cd9ba-c6fc-4309-baa6-05c4fffd5012 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.101836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e9d00-ef70-486e-ad86-f264860760f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.102379 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e9d00-ef70-486e-ad86-f264860760f9 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.102666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e9d00-ef70-486e-ad86-f264860760f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.102939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e9d00-ef70-486e-ad86-f264860760f9 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.103670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e9d00-ef70-486e-ad86-f264860760f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:08.103960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e9d00-ef70-486e-ad86-f264860760f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:08.105341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e9d00-ef70-486e-ad86-f264860760f9 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:08.114824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 61/116] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:08.116140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9ae29-3e51-44ef-ab8b-c5d0149bdf07 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.116436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9ae29-3e51-44ef-ab8b-c5d0149bdf07 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.116713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9ae29-3e51-44ef-ab8b-c5d0149bdf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.138960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-adb9ae29-3e51-44ef-ab8b-c5d0149bdf07 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:08.140329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-adb9ae29-3e51-44ef-ab8b-c5d0149bdf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:08.147055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ca9dd-3653-41a1-bfb7-7b42e3d3f2f5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.159533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:08.184709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ac785-9fae-401d-9192-e4f7d5f89dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.185890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ac785-9fae-401d-9192-e4f7d5f89dad None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.186177 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ac785-9fae-401d-9192-e4f7d5f89dad None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.186450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ac785-9fae-401d-9192-e4f7d5f89dad None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.187146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ac785-9fae-401d-9192-e4f7d5f89dad None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:08.187417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ac785-9fae-401d-9192-e4f7d5f89dad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:08.188798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ac785-9fae-401d-9192-e4f7d5f89dad None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:08.198303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 56/117] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:08.202788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:08.203879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ac785-9fae-401d-9192-e4f7d5f89dad None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.214782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 62/118] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:08.216137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9ae29-3e51-44ef-ab8b-c5d0149bdf07 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.315919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ac785-9fae-401d-9192-e4f7d5f89dad None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.322335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9ae29-3e51-44ef-ab8b-c5d0149bdf07 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.349021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951696fe-c1d7-4351-a9f8-90c99c2f0d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.349659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951696fe-c1d7-4351-a9f8-90c99c2f0d29 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.349973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951696fe-c1d7-4351-a9f8-90c99c2f0d29 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.350249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951696fe-c1d7-4351-a9f8-90c99c2f0d29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.360616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3106a7-166c-4c69-b21e-5d6d747fade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.361273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951696fe-c1d7-4351-a9f8-90c99c2f0d29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:08.361943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3106a7-166c-4c69-b21e-5d6d747fade0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.362367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3106a7-166c-4c69-b21e-5d6d747fade0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.362782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3106a7-166c-4c69-b21e-5d6d747fade0 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.364051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3106a7-166c-4c69-b21e-5d6d747fade0 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:08.365662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3106a7-166c-4c69-b21e-5d6d747fade0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-951696fe-c1d7-4351-a9f8-90c99c2f0d29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:08.366433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:08.367439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:08.368384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:08.370300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 63/119] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:08.371205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3106a7-166c-4c69-b21e-5d6d747fade0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.375345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 57/120] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:08.376282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951696fe-c1d7-4351-a9f8-90c99c2f0d29 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.416336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:08.439941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:08.487827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3106a7-166c-4c69-b21e-5d6d747fade0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.490260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951696fe-c1d7-4351-a9f8-90c99c2f0d29 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.521745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7055811-a602-4a03-907c-74a6bc721230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.522402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7055811-a602-4a03-907c-74a6bc721230 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.522688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7055811-a602-4a03-907c-74a6bc721230 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.522982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7055811-a602-4a03-907c-74a6bc721230 None admin] PATH_INFO: `/v3/users/cf123039ca4549b09c08e01db87afcb3` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.524243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064ca91-8284-4a7e-ad2f-13043833daed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.524874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064ca91-8284-4a7e-ad2f-13043833daed None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.525150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064ca91-8284-4a7e-ad2f-13043833daed None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.525423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064ca91-8284-4a7e-ad2f-13043833daed None admin] PATH_INFO: `/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.526650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064ca91-8284-4a7e-ad2f-13043833daed None admin] RBAC: Authorizing `identity:get_user(user_id=8502ad6c48f34f22831d3b9a1d5e8e5d)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:08.528098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064ca91-8284-4a7e-ad2f-13043833daed None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:08.535011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 58/121] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:08.535011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064ca91-8284-4a7e-ad2f-13043833daed None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.543250 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7055811-a602-4a03-907c-74a6bc721230 None admin] RBAC: Authorizing `identity:get_user(user_id=cf123039ca4549b09c08e01db87afcb3)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:08.546125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7055811-a602-4a03-907c-74a6bc721230 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:08.552734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 64/122] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/users/cf123039ca4549b09c08e01db87afcb3 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:08.552734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7055811-a602-4a03-907c-74a6bc721230 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.651574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:08.654981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064ca91-8284-4a7e-ad2f-13043833daed None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.661351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7055811-a602-4a03-907c-74a6bc721230 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.679081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:08.690761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d723-3548-4ad9-805d-0b71e9bad1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.691409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d723-3548-4ad9-805d-0b71e9bad1e8 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.691679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d723-3548-4ad9-805d-0b71e9bad1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.691958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0d723-3548-4ad9-805d-0b71e9bad1e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.697170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c5456-7343-4a36-b5cb-17f0e6ac102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.697856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c5456-7343-4a36-b5cb-17f0e6ac102b None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.698140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c5456-7343-4a36-b5cb-17f0e6ac102b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.698414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c5456-7343-4a36-b5cb-17f0e6ac102b None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/users/3b9f97788d554a69b9c325c4e2da6890/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.700317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c5456-7343-4a36-b5cb-17f0e6ac102b None admin] RBAC: Authorizing `identity:create_grant(project_id=b5bf4929d4984d40a41eaf5bd240ffd0, user_id=3b9f97788d554a69b9c325c4e2da6890, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:08.702821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c5456-7343-4a36-b5cb-17f0e6ac102b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:08.703238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0d723-3548-4ad9-805d-0b71e9bad1e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-92d0d723-3548-4ad9-805d-0b71e9bad1e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:08.707943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:08.709035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 59/123] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:08.710227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d723-3548-4ad9-805d-0b71e9bad1e8 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.788412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 65/124] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:40:08 2022] PUT /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/users/3b9f97788d554a69b9c325c4e2da6890/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:08.797995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c5456-7343-4a36-b5cb-17f0e6ac102b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.835478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0d723-3548-4ad9-805d-0b71e9bad1e8 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.889328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946449a-db88-4a50-96f1-f9d1b414ffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.890007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946449a-db88-4a50-96f1-f9d1b414ffe6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.890316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946449a-db88-4a50-96f1-f9d1b414ffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.891248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946449a-db88-4a50-96f1-f9d1b414ffe6 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.891942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946449a-db88-4a50-96f1-f9d1b414ffe6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:08.892249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946449a-db88-4a50-96f1-f9d1b414ffe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:08.894221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946449a-db88-4a50-96f1-f9d1b414ffe6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:08.910976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 60/125] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:08.910976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946449a-db88-4a50-96f1-f9d1b414ffe6 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:08.922507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c5456-7343-4a36-b5cb-17f0e6ac102b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:08.948500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:08.964368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014d5e3-bb50-4ef9-b151-e709cc06882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:08.965013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014d5e3-bb50-4ef9-b151-e709cc06882f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:08.965297 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014d5e3-bb50-4ef9-b151-e709cc06882f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:08.965573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014d5e3-bb50-4ef9-b151-e709cc06882f None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:08.967317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014d5e3-bb50-4ef9-b151-e709cc06882f None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:08.968779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014d5e3-bb50-4ef9-b151-e709cc06882f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:08.971269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 66/126] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:08.972450 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014d5e3-bb50-4ef9-b151-e709cc06882f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:09.022355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:09.053952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946449a-db88-4a50-96f1-f9d1b414ffe6 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:09.080702 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014d5e3-bb50-4ef9-b151-e709cc06882f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:09.107581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc92aa8-ceb6-42ae-ba1b-568ac8823935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:09.108730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc92aa8-ceb6-42ae-ba1b-568ac8823935 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.109448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc92aa8-ceb6-42ae-ba1b-568ac8823935 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.109927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc92aa8-ceb6-42ae-ba1b-568ac8823935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.118968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f1083-81b9-469a-921c-eca2097b5521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:09.119706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f1083-81b9-469a-921c-eca2097b5521 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.120086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f1083-81b9-469a-921c-eca2097b5521 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.120432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f1083-81b9-469a-921c-eca2097b5521 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.121222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f1083-81b9-469a-921c-eca2097b5521 None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:09.121598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f1083-81b9-469a-921c-eca2097b5521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:09.123786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f1083-81b9-469a-921c-eca2097b5521 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:09.132399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 67/127] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:09.137560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f1083-81b9-469a-921c-eca2097b5521 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:09.206562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc92aa8-ceb6-42ae-ba1b-568ac8823935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:09.207991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc92aa8-ceb6-42ae-ba1b-568ac8823935 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:09.236429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:09.258823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f1083-81b9-469a-921c-eca2097b5521 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:09.268793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:09.296324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f741ab-ac6f-44e2-82f1-da63682db609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:09.297056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f741ab-ac6f-44e2-82f1-da63682db609 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.297471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f741ab-ac6f-44e2-82f1-da63682db609 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.297855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f741ab-ac6f-44e2-82f1-da63682db609 None admin] PATH_INFO: `/v3/users/3b9f97788d554a69b9c325c4e2da6890` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.299159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f741ab-ac6f-44e2-82f1-da63682db609 None admin] RBAC: Authorizing `identity:get_user(user_id=3b9f97788d554a69b9c325c4e2da6890)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:09.301079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f741ab-ac6f-44e2-82f1-da63682db609 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:09.302463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 68/128] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:09 2022] GET /identity/v3/users/3b9f97788d554a69b9c325c4e2da6890 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:09.308494 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f741ab-ac6f-44e2-82f1-da63682db609 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:09.344327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 61/129] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:09.344327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc92aa8-ceb6-42ae-ba1b-568ac8823935 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:09.421086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f741ab-ac6f-44e2-82f1-da63682db609 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:09.444627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc92aa8-ceb6-42ae-ba1b-568ac8823935 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:09.456160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f46380-71f5-47f7-826e-57a94416a264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:09.456942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f46380-71f5-47f7-826e-57a94416a264 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.457172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f46380-71f5-47f7-826e-57a94416a264 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.457446 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f46380-71f5-47f7-826e-57a94416a264 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/users/8502ad6c48f34f22831d3b9a1d5e8e5d/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.459630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f46380-71f5-47f7-826e-57a94416a264 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5bf4929d4984d40a41eaf5bd240ffd0, user_id=8502ad6c48f34f22831d3b9a1d5e8e5d, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:09.461198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f46380-71f5-47f7-826e-57a94416a264 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:09.468259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:09.479621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd446df-757d-4a3c-ad8a-c1194169c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:09.480254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd446df-757d-4a3c-ad8a-c1194169c744 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.480539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd446df-757d-4a3c-ad8a-c1194169c744 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.480840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd446df-757d-4a3c-ad8a-c1194169c744 None admin] PATH_INFO: `/v3/users/3b9f97788d554a69b9c325c4e2da6890` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.482682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd446df-757d-4a3c-ad8a-c1194169c744 None admin] RBAC: Authorizing `identity:get_user(user_id=3b9f97788d554a69b9c325c4e2da6890)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:09.484199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd446df-757d-4a3c-ad8a-c1194169c744 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:09.485815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 62/130] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:09 2022] GET /identity/v3/users/3b9f97788d554a69b9c325c4e2da6890 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:09.485815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd446df-757d-4a3c-ad8a-c1194169c744 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:09.560744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 69/131] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:40:09 2022] PUT /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/users/8502ad6c48f34f22831d3b9a1d5e8e5d/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:09.560744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79da11-3409-419e-a84e-acc07e6f761c None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.560744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79da11-3409-419e-a84e-acc07e6f761c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.560744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79da11-3409-419e-a84e-acc07e6f761c None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.562134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 70/132] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:09.562943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79da11-3409-419e-a84e-acc07e6f761c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:09.615670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:09.622003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd446df-757d-4a3c-ad8a-c1194169c744 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:09.674852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa87877-4060-4447-99e8-801a27dceef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:09.676538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa87877-4060-4447-99e8-801a27dceef6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.677433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa87877-4060-4447-99e8-801a27dceef6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.677916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa87877-4060-4447-99e8-801a27dceef6 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.679398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa87877-4060-4447-99e8-801a27dceef6 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:09.682187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa87877-4060-4447-99e8-801a27dceef6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:09.685957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 63/133] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:09 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:09.687072 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa87877-4060-4447-99e8-801a27dceef6 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:09.701339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79da11-3409-419e-a84e-acc07e6f761c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:09.744664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d1739-dde0-4119-bf32-24fd17310fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:09.746647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d1739-dde0-4119-bf32-24fd17310fd5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.747081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d1739-dde0-4119-bf32-24fd17310fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.748452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d1739-dde0-4119-bf32-24fd17310fd5 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.749957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d1739-dde0-4119-bf32-24fd17310fd5 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:09.751710 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d1739-dde0-4119-bf32-24fd17310fd5 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:09.754120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 71/134] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:09 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:09.758096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7aae0-29cd-4068-9e35-de6a1c48a507 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.758557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7aae0-29cd-4068-9e35-de6a1c48a507 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.758841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7aae0-29cd-4068-9e35-de6a1c48a507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.777303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-cfe7aae0-29cd-4068-9e35-de6a1c48a507 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:09.778581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-cfe7aae0-29cd-4068-9e35-de6a1c48a507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:09.782896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:09.833825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa87877-4060-4447-99e8-801a27dceef6 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:09.854319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 72/135] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:09.855342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7aae0-29cd-4068-9e35-de6a1c48a507 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:09.878523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ece90d-4edf-4c53-8642-5751cb625a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:09.879985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ece90d-4edf-4c53-8642-5751cb625a9a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:09.880724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ece90d-4edf-4c53-8642-5751cb625a9a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:09.881193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ece90d-4edf-4c53-8642-5751cb625a9a None admin] PATH_INFO: `/v3/projects/ccf3799e8d9e4dcb8a61bea731cb301d` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:09.883217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ece90d-4edf-4c53-8642-5751cb625a9a None admin] RBAC: Authorizing `identity:get_project(project_id=ccf3799e8d9e4dcb8a61bea731cb301d)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:09.885412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ece90d-4edf-4c53-8642-5751cb625a9a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:09.888441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 64/136] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:09 2022] GET /identity/v3/projects/ccf3799e8d9e4dcb8a61bea731cb301d => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:09.890011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ece90d-4edf-4c53-8642-5751cb625a9a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:09.894903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:09.968546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7aae0-29cd-4068-9e35-de6a1c48a507 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:10.006533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a62564-e971-4493-a1a5-d608ef69cc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:10.007499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a62564-e971-4493-a1a5-d608ef69cc8f None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:10.007953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a62564-e971-4493-a1a5-d608ef69cc8f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:10.008413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a62564-e971-4493-a1a5-d608ef69cc8f None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/users/3b9f97788d554a69b9c325c4e2da6890/roles/d7d75f4c26eb44689ec71efb8b63e534` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:10.010799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a62564-e971-4493-a1a5-d608ef69cc8f None admin] RBAC: Authorizing `identity:create_grant(project_id=b5bf4929d4984d40a41eaf5bd240ffd0, user_id=3b9f97788d554a69b9c325c4e2da6890, role_id=d7d75f4c26eb44689ec71efb8b63e534)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:10.012440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a62564-e971-4493-a1a5-d608ef69cc8f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:10.026506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ece90d-4edf-4c53-8642-5751cb625a9a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:10.053931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:10.063254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef4e0-322f-4c15-9f26-274e19698fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:10.063868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef4e0-322f-4c15-9f26-274e19698fee None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:10.064154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef4e0-322f-4c15-9f26-274e19698fee None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:10.064498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aef4e0-322f-4c15-9f26-274e19698fee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:10.065231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aef4e0-322f-4c15-9f26-274e19698fee None admin] RBAC: Adding query filter params (role.id=63d2a6efa7784a7eab9594616b3e8978, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0, user.id=cf123039ca4549b09c08e01db87afcb3) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:10.065606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aef4e0-322f-4c15-9f26-274e19698fee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:10.067316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aef4e0-322f-4c15-9f26-274e19698fee None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:10.072983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 65/137] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:09 2022] GET /identity/v3/role_assignments?user.id=cf123039ca4549b09c08e01db87afcb3&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=63d2a6efa7784a7eab9594616b3e8978 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:10.073692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef4e0-322f-4c15-9f26-274e19698fee None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:10.097691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 73/138] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:40:09 2022] PUT /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/users/3b9f97788d554a69b9c325c4e2da6890/roles/d7d75f4c26eb44689ec71efb8b63e534 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:10.098840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a62564-e971-4493-a1a5-d608ef69cc8f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:10.249819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:10.255430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a62564-e971-4493-a1a5-d608ef69cc8f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:10.315406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744e8fb-4e59-4d14-8dff-19dd5700f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:10.316200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744e8fb-4e59-4d14-8dff-19dd5700f3d4 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:10.316520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744e8fb-4e59-4d14-8dff-19dd5700f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:10.316799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744e8fb-4e59-4d14-8dff-19dd5700f3d4 None admin] PATH_INFO: `/v3/projects/ccf3799e8d9e4dcb8a61bea731cb301d/users/3b9f97788d554a69b9c325c4e2da6890/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:10.323402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744e8fb-4e59-4d14-8dff-19dd5700f3d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccf3799e8d9e4dcb8a61bea731cb301d, user_id=3b9f97788d554a69b9c325c4e2da6890, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:10.324249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aef4e0-322f-4c15-9f26-274e19698fee None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:10.325609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744e8fb-4e59-4d14-8dff-19dd5700f3d4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:10.386130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d29b3f-c151-4b9f-993d-93e08681e5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:10.386837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d29b3f-c151-4b9f-993d-93e08681e5ea None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:10.387252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d29b3f-c151-4b9f-993d-93e08681e5ea None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:10.387527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d29b3f-c151-4b9f-993d-93e08681e5ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:10.390668 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d29b3f-c151-4b9f-993d-93e08681e5ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:10.392266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d29b3f-c151-4b9f-993d-93e08681e5ea None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:10.394287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 66/139] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:10.400013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d29b3f-c151-4b9f-993d-93e08681e5ea None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:10.410332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 74/140] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:40:10 2022] PUT /identity/v3/projects/ccf3799e8d9e4dcb8a61bea731cb301d/users/3b9f97788d554a69b9c325c4e2da6890/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:10.471997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:10.523721 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d29b3f-c151-4b9f-993d-93e08681e5ea None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:10.567746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46913147-87d7-4a61-9354-78ae346bd6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:10.568350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46913147-87d7-4a61-9354-78ae346bd6c4 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:10.568637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46913147-87d7-4a61-9354-78ae346bd6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:10.568912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46913147-87d7-4a61-9354-78ae346bd6c4 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:10.569597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46913147-87d7-4a61-9354-78ae346bd6c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:10.571198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46913147-87d7-4a61-9354-78ae346bd6c4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:10.675490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 67/141] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:10 2022] POST /identity/v3/users => generated 283 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:11.190142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed81b2d-5f1a-415c-a332-f8ff459102c0 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:11.190921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed81b2d-5f1a-415c-a332-f8ff459102c0 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:11.190921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed81b2d-5f1a-415c-a332-f8ff459102c0 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:11.192025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 75/142] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:11.203468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77b046-07e7-449b-bfd2-3e2ffe34d9fe None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:11.203733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77b046-07e7-449b-bfd2-3e2ffe34d9fe None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:11.204017 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77b046-07e7-449b-bfd2-3e2ffe34d9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:11.230799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ef77b046-07e7-449b-bfd2-3e2ffe34d9fe None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:11.232509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ef77b046-07e7-449b-bfd2-3e2ffe34d9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:11.322200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 68/143] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:11.363223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed81b2d-5f1a-415c-a332-f8ff459102c0 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:11.479290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed81b2d-5f1a-415c-a332-f8ff459102c0 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:11.515004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3ea73-23d4-4261-b4bc-e269a70bdc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:11.515637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3ea73-23d4-4261-b4bc-e269a70bdc12 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:11.515917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3ea73-23d4-4261-b4bc-e269a70bdc12 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:11.516190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3ea73-23d4-4261-b4bc-e269a70bdc12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:11.533134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3ea73-23d4-4261-b4bc-e269a70bdc12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-ace3ea73-23d4-4261-b4bc-e269a70bdc12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:11.537853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:11.539874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:11.541753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 76/144] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:11.549810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77b046-07e7-449b-bfd2-3e2ffe34d9fe None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:11.608644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:11.681475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77b046-07e7-449b-bfd2-3e2ffe34d9fe None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:11.716842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d3406-9c2e-4e21-8ff8-343e96f8ea86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:11.717455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d3406-9c2e-4e21-8ff8-343e96f8ea86 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:11.717737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d3406-9c2e-4e21-8ff8-343e96f8ea86 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:11.718037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d3406-9c2e-4e21-8ff8-343e96f8ea86 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:11.718707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d3406-9c2e-4e21-8ff8-343e96f8ea86 None admin] RBAC: Adding query filter params (name=member) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:11.719010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d3406-9c2e-4e21-8ff8-343e96f8ea86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:11.720404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d3406-9c2e-4e21-8ff8-343e96f8ea86 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:11.734006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 69/145] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:11.740295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3ea73-23d4-4261-b4bc-e269a70bdc12 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:11.762868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:11.869572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3ea73-23d4-4261-b4bc-e269a70bdc12 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:11.913497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b87b66-53a3-4318-bfb4-aedde259c0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:11.914166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b87b66-53a3-4318-bfb4-aedde259c0c9 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:11.914447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b87b66-53a3-4318-bfb4-aedde259c0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:11.914556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c9e92-9f7d-4114-829d-6dbf827d0967 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:11.914732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b87b66-53a3-4318-bfb4-aedde259c0c9 None admin] PATH_INFO: `/v3/users/3b9f97788d554a69b9c325c4e2da6890` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:11.914835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c9e92-9f7d-4114-829d-6dbf827d0967 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:11.916328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c9e92-9f7d-4114-829d-6dbf827d0967 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:11.916463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b87b66-53a3-4318-bfb4-aedde259c0c9 None admin] RBAC: Authorizing `identity:get_user(user_id=3b9f97788d554a69b9c325c4e2da6890)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:11.916548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 70/146] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:11.917814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b87b66-53a3-4318-bfb4-aedde259c0c9 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:11.919466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 77/147] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:11 2022] GET /identity/v3/users/3b9f97788d554a69b9c325c4e2da6890 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:11.924922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c9e92-9f7d-4114-829d-6dbf827d0967 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:11.927361 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b890d81-8553-49e5-9f54-f40ca532be81 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:11.927649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b890d81-8553-49e5-9f54-f40ca532be81 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:11.927923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b890d81-8553-49e5-9f54-f40ca532be81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:11.951542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-3b890d81-8553-49e5-9f54-f40ca532be81 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:11.952999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3b890d81-8553-49e5-9f54-f40ca532be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:12.059213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 78/148] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:12.098872 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b890d81-8553-49e5-9f54-f40ca532be81 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:12.120646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:12.171468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c9e92-9f7d-4114-829d-6dbf827d0967 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:12.183761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:12.243480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b890d81-8553-49e5-9f54-f40ca532be81 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:12.246115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7400c-cf54-4d9b-b283-9c1ad8232fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:12.246735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7400c-cf54-4d9b-b283-9c1ad8232fac None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:12.247043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7400c-cf54-4d9b-b283-9c1ad8232fac None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:12.247319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7400c-cf54-4d9b-b283-9c1ad8232fac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:12.289668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a41b7-e952-447b-95f8-840b536ee06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:12.290332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a41b7-e952-447b-95f8-840b536ee06a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:12.290609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a41b7-e952-447b-95f8-840b536ee06a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:12.290878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a41b7-e952-447b-95f8-840b536ee06a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:12.302815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a41b7-e952-447b-95f8-840b536ee06a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-7f5a41b7-e952-447b-95f8-840b536ee06a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:12.307472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:12.308432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:12.309482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 79/149] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:40:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:12.321714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a41b7-e952-447b-95f8-840b536ee06a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:12.373099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7400c-cf54-4d9b-b283-9c1ad8232fac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:12.374643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7400c-cf54-4d9b-b283-9c1ad8232fac None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:12.422637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:12.474391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a41b7-e952-447b-95f8-840b536ee06a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:12.566237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 71/150] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:12.572310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7400c-cf54-4d9b-b283-9c1ad8232fac None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:12.590571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e9a9a-45ea-42b9-95e8-0f5b340f9491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:12.592605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e9a9a-45ea-42b9-95e8-0f5b340f9491 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:12.592605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e9a9a-45ea-42b9-95e8-0f5b340f9491 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:12.592605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e9a9a-45ea-42b9-95e8-0f5b340f9491 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:12.592957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e9a9a-45ea-42b9-95e8-0f5b340f9491 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:12.593256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e9a9a-45ea-42b9-95e8-0f5b340f9491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:12.597812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e9a9a-45ea-42b9-95e8-0f5b340f9491 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:12.605094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:12.612684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 80/151] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:12.618059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e9a9a-45ea-42b9-95e8-0f5b340f9491 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:12.738922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7400c-cf54-4d9b-b283-9c1ad8232fac None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:12.790257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db764d36-fef8-4ae5-9456-bbfe2932cd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:12.791352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db764d36-fef8-4ae5-9456-bbfe2932cd55 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:12.791817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db764d36-fef8-4ae5-9456-bbfe2932cd55 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:12.792268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db764d36-fef8-4ae5-9456-bbfe2932cd55 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:12.794761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db764d36-fef8-4ae5-9456-bbfe2932cd55 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:12.796431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db764d36-fef8-4ae5-9456-bbfe2932cd55 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:12.798327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 72/152] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:12 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:12.804432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db764d36-fef8-4ae5-9456-bbfe2932cd55 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:12.916819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:12.930735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db764d36-fef8-4ae5-9456-bbfe2932cd55 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:12.968372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e9a9a-45ea-42b9-95e8-0f5b340f9491 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:13.000010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89857-6491-449a-bebe-4d80c7b1e2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:13.000795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89857-6491-449a-bebe-4d80c7b1e2ed None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:13.001072 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89857-6491-449a-bebe-4d80c7b1e2ed None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:13.001441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89857-6491-449a-bebe-4d80c7b1e2ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:13.002203 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb89857-6491-449a-bebe-4d80c7b1e2ed None admin] RBAC: Adding query filter params (role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0, user.id=3b9f97788d554a69b9c325c4e2da6890) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:13.002510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb89857-6491-449a-bebe-4d80c7b1e2ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:13.003943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb89857-6491-449a-bebe-4d80c7b1e2ed None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:13.015815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 73/153] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:12 2022] GET /identity/v3/role_assignments?user.id=3b9f97788d554a69b9c325c4e2da6890&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:13.034596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ac3d7-18a4-4241-883f-5a7af152a8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:13.035262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ac3d7-18a4-4241-883f-5a7af152a8f3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:13.035541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ac3d7-18a4-4241-883f-5a7af152a8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:13.035793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ac3d7-18a4-4241-883f-5a7af152a8f3 None admin] PATH_INFO: `/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:13.037046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ac3d7-18a4-4241-883f-5a7af152a8f3 None admin] RBAC: Authorizing `identity:get_user(user_id=8502ad6c48f34f22831d3b9a1d5e8e5d)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:13.038672 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ac3d7-18a4-4241-883f-5a7af152a8f3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:13.040607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 81/154] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:12 2022] GET /identity/v3/users/8502ad6c48f34f22831d3b9a1d5e8e5d => generated 247 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:13.046598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89857-6491-449a-bebe-4d80c7b1e2ed None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:13.198782 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89857-6491-449a-bebe-4d80c7b1e2ed None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:13.236115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75f02-a94d-4fd3-a9d4-e0f575911218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:13.236729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75f02-a94d-4fd3-a9d4-e0f575911218 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:13.237012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75f02-a94d-4fd3-a9d4-e0f575911218 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:13.237324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75f02-a94d-4fd3-a9d4-e0f575911218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:13.246359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:13.312176 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75f02-a94d-4fd3-a9d4-e0f575911218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:13.313689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75f02-a94d-4fd3-a9d4-e0f575911218 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:13.419619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 74/155] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:13.425567 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ac3d7-18a4-4241-883f-5a7af152a8f3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:13.430092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:13.571592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ac3d7-18a4-4241-883f-5a7af152a8f3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:13.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fdb90-94e8-4e33-9d6d-5c24dc9a890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:13.615271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fdb90-94e8-4e33-9d6d-5c24dc9a890b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:13.615547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fdb90-94e8-4e33-9d6d-5c24dc9a890b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:13.615814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fdb90-94e8-4e33-9d6d-5c24dc9a890b None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:13.623045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fdb90-94e8-4e33-9d6d-5c24dc9a890b None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:13.624498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fdb90-94e8-4e33-9d6d-5c24dc9a890b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:13.626404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 82/156] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:13 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:13.632526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75f02-a94d-4fd3-a9d4-e0f575911218 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:13.706272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:13.769761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75f02-a94d-4fd3-a9d4-e0f575911218 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:13.810725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372686f9-2670-4401-a05c-33efa463c113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:13.811712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372686f9-2670-4401-a05c-33efa463c113 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:13.812259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372686f9-2670-4401-a05c-33efa463c113 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:13.812735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372686f9-2670-4401-a05c-33efa463c113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:13.813663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372686f9-2670-4401-a05c-33efa463c113 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0, user.id=8502ad6c48f34f22831d3b9a1d5e8e5d) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:13.814209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372686f9-2670-4401-a05c-33efa463c113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:13.815886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372686f9-2670-4401-a05c-33efa463c113 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:13.822655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 75/157] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:13 2022] GET /identity/v3/role_assignments?user.id=8502ad6c48f34f22831d3b9a1d5e8e5d&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:13.839270 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582667f1-8dec-498c-87f8-5a80e3082f06 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:13.839566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582667f1-8dec-498c-87f8-5a80e3082f06 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:13.839808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582667f1-8dec-498c-87f8-5a80e3082f06 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:13.846174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 83/158] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:13.848063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a437e4c-ad3e-4074-8109-a6f1e23ae595 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:13.848576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a437e4c-ad3e-4074-8109-a6f1e23ae595 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:13.849056 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a437e4c-ad3e-4074-8109-a6f1e23ae595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:13.875094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-7a437e4c-ad3e-4074-8109-a6f1e23ae595 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:13.876687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7a437e4c-ad3e-4074-8109-a6f1e23ae595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:13.967301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 76/159] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:14.002357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582667f1-8dec-498c-87f8-5a80e3082f06 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:14.106808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:14.138115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b8d99-4ee8-47b6-8fa1-bcff65e041cf None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.138630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b8d99-4ee8-47b6-8fa1-bcff65e041cf None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.139054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b8d99-4ee8-47b6-8fa1-bcff65e041cf None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.140571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 77/160] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:40:14.146891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450a2bf-c822-47b4-8d35-c4b13d96c905 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.147398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450a2bf-c822-47b4-8d35-c4b13d96c905 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.147820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450a2bf-c822-47b4-8d35-c4b13d96c905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.171281 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582667f1-8dec-498c-87f8-5a80e3082f06 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:14.173776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6450a2bf-c822-47b4-8d35-c4b13d96c905 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:14.175923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6450a2bf-c822-47b4-8d35-c4b13d96c905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:14.233886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:14.238879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb85b3-2138-4043-9e1a-ae1c00966adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:14.239957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb85b3-2138-4043-9e1a-ae1c00966adf None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.241565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb85b3-2138-4043-9e1a-ae1c00966adf None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.242035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb85b3-2138-4043-9e1a-ae1c00966adf None admin] PATH_INFO: `/v3/roles/member` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.267405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb85b3-2138-4043-9e1a-ae1c00966adf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-c7bb85b3-2138-4043-9e1a-ae1c00966adf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:14.279060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:14.280272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:14.281208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:14.289437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 84/161] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:14.290960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f988e7d-681b-4479-b38e-06e60c47a20a None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.292451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f988e7d-681b-4479-b38e-06e60c47a20a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.294339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f988e7d-681b-4479-b38e-06e60c47a20a None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.295824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 85/162] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:14.297686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f988e7d-681b-4479-b38e-06e60c47a20a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:14.333716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 78/163] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:14.336125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe33c-0af7-49a0-b942-32670f26a2d5 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.336713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe33c-0af7-49a0-b942-32670f26a2d5 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.338315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe33c-0af7-49a0-b942-32670f26a2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.370596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-928fe33c-0af7-49a0-b942-32670f26a2d5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:14.372168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-928fe33c-0af7-49a0-b942-32670f26a2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:14.450844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 79/164] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:14.452265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fe33c-0af7-49a0-b942-32670f26a2d5 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:14.503531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f988e7d-681b-4479-b38e-06e60c47a20a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:14.547109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:14.563416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2f455-4d0e-499e-b3e7-fe984c1e1fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:14.569194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2f455-4d0e-499e-b3e7-fe984c1e1fcc None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.569194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2f455-4d0e-499e-b3e7-fe984c1e1fcc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.569194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2f455-4d0e-499e-b3e7-fe984c1e1fcc None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.569194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2f455-4d0e-499e-b3e7-fe984c1e1fcc None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:14.569194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2f455-4d0e-499e-b3e7-fe984c1e1fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:14.569549 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fe33c-0af7-49a0-b942-32670f26a2d5 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:14.569630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2f455-4d0e-499e-b3e7-fe984c1e1fcc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:14.569697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:14.584037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 86/165] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:14.585238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2f455-4d0e-499e-b3e7-fe984c1e1fcc None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:14.622480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02568e5e-fc9e-41b5-bb2a-2cdae8b43042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:14.623469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02568e5e-fc9e-41b5-bb2a-2cdae8b43042 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.623975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02568e5e-fc9e-41b5-bb2a-2cdae8b43042 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.624437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02568e5e-fc9e-41b5-bb2a-2cdae8b43042 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.636340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02568e5e-fc9e-41b5-bb2a-2cdae8b43042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-02568e5e-fc9e-41b5-bb2a-2cdae8b43042 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:14.642008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:14.642873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:14.643746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:14.645857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 80/166] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:14.646815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02568e5e-fc9e-41b5-bb2a-2cdae8b43042 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:14.754038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02568e5e-fc9e-41b5-bb2a-2cdae8b43042 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:14.758532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2f455-4d0e-499e-b3e7-fe984c1e1fcc None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:14.791105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6345e4c-899f-479b-bacd-7442602e2dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:14.792412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6345e4c-899f-479b-bacd-7442602e2dd3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.792923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6345e4c-899f-479b-bacd-7442602e2dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.793482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6345e4c-899f-479b-bacd-7442602e2dd3 None admin] PATH_INFO: `/v3/users/3b9f97788d554a69b9c325c4e2da6890` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.795471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6345e4c-899f-479b-bacd-7442602e2dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=3b9f97788d554a69b9c325c4e2da6890)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:14.797441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6345e4c-899f-479b-bacd-7442602e2dd3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:14.799430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 81/167] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:14 2022] GET /identity/v3/users/3b9f97788d554a69b9c325c4e2da6890 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:14.800646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6345e4c-899f-479b-bacd-7442602e2dd3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:14.823701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:14.845274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281465f3-6bc6-479c-9351-44e473a91e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:14.845601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281465f3-6bc6-479c-9351-44e473a91e08 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.845813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281465f3-6bc6-479c-9351-44e473a91e08 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.846092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281465f3-6bc6-479c-9351-44e473a91e08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.871317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281465f3-6bc6-479c-9351-44e473a91e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-281465f3-6bc6-479c-9351-44e473a91e08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:14.876835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:14.877709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:14.879096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 87/168] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:14.886643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281465f3-6bc6-479c-9351-44e473a91e08 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:14.915910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6345e4c-899f-479b-bacd-7442602e2dd3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:14.960511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e036b-bf3d-4400-a8e8-1d1f4c98f57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:14.961567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e036b-bf3d-4400-a8e8-1d1f4c98f57a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:14.962075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e036b-bf3d-4400-a8e8-1d1f4c98f57a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:14.962568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e036b-bf3d-4400-a8e8-1d1f4c98f57a None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:14.963647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e036b-bf3d-4400-a8e8-1d1f4c98f57a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:14.964525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e036b-bf3d-4400-a8e8-1d1f4c98f57a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:14.967496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e036b-bf3d-4400-a8e8-1d1f4c98f57a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:14.976813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 82/169] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:14.978053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e036b-bf3d-4400-a8e8-1d1f4c98f57a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:14.986502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:15.050167 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281465f3-6bc6-479c-9351-44e473a91e08 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:15.115533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d23c4a-3466-4ac8-b204-e98cb0862a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:15.116150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d23c4a-3466-4ac8-b204-e98cb0862a8b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.116423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d23c4a-3466-4ac8-b204-e98cb0862a8b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.116689 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d23c4a-3466-4ac8-b204-e98cb0862a8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.135360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:15.175511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e036b-bf3d-4400-a8e8-1d1f4c98f57a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:15.227961 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d23c4a-3466-4ac8-b204-e98cb0862a8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:15.229288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d23c4a-3466-4ac8-b204-e98cb0862a8b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:15.237038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aac600-c181-411d-8002-b8c7c72c5e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:15.238101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aac600-c181-411d-8002-b8c7c72c5e16 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.238609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aac600-c181-411d-8002-b8c7c72c5e16 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.239085 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aac600-c181-411d-8002-b8c7c72c5e16 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.240046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aac600-c181-411d-8002-b8c7c72c5e16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:15.240561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aac600-c181-411d-8002-b8c7c72c5e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:15.242298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aac600-c181-411d-8002-b8c7c72c5e16 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:15.252012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 83/170] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:15.253098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aac600-c181-411d-8002-b8c7c72c5e16 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:15.282491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:15.336534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 88/171] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:15.338079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff61e-2949-43ed-8830-1023695fae2a None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.338359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff61e-2949-43ed-8830-1023695fae2a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.338629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff61e-2949-43ed-8830-1023695fae2a None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.341584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 89/172] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:15.343260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff61e-2949-43ed-8830-1023695fae2a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:15.373037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aac600-c181-411d-8002-b8c7c72c5e16 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:15.411332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175df14a-d628-408c-8f2c-c7da97dfd26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:15.411995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175df14a-d628-408c-8f2c-c7da97dfd26f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.412355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175df14a-d628-408c-8f2c-c7da97dfd26f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.412624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175df14a-d628-408c-8f2c-c7da97dfd26f None admin] PATH_INFO: `/v3/users/3b9f97788d554a69b9c325c4e2da6890` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.413772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175df14a-d628-408c-8f2c-c7da97dfd26f None admin] RBAC: Authorizing `identity:get_user(user_id=3b9f97788d554a69b9c325c4e2da6890)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:15.415311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175df14a-d628-408c-8f2c-c7da97dfd26f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:15.421859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 84/173] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:15 2022] GET /identity/v3/users/3b9f97788d554a69b9c325c4e2da6890 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:15.421859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500b960-2620-4eee-a94d-0377729480af None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.421859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500b960-2620-4eee-a94d-0377729480af None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.421859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500b960-2620-4eee-a94d-0377729480af None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.421859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 85/174] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:15.421859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500b960-2620-4eee-a94d-0377729480af None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:15.443779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:15.450977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff61e-2949-43ed-8830-1023695fae2a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:15.487492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0c566-1653-4802-914f-2d082f9116c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:15.488118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0c566-1653-4802-914f-2d082f9116c3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.488390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0c566-1653-4802-914f-2d082f9116c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.488657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0c566-1653-4802-914f-2d082f9116c3 None admin] PATH_INFO: `/v3/users/cf123039ca4549b09c08e01db87afcb3` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.489773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0c566-1653-4802-914f-2d082f9116c3 None admin] RBAC: Authorizing `identity:get_user(user_id=cf123039ca4549b09c08e01db87afcb3)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:15.491232 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0c566-1653-4802-914f-2d082f9116c3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:15.492875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 90/175] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:15 2022] GET /identity/v3/users/cf123039ca4549b09c08e01db87afcb3 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:15.493968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a582f-e12d-4202-8568-db526dbd2380 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.494174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a582f-e12d-4202-8568-db526dbd2380 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.494440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a582f-e12d-4202-8568-db526dbd2380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.513316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-134a582f-e12d-4202-8568-db526dbd2380 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:15.514523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-134a582f-e12d-4202-8568-db526dbd2380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:15.528336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:15.541304 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500b960-2620-4eee-a94d-0377729480af None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:15.575673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b82fc-f055-4e07-b7d3-de1c58851333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:15.576334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b82fc-f055-4e07-b7d3-de1c58851333 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.576927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b82fc-f055-4e07-b7d3-de1c58851333 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.577202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b82fc-f055-4e07-b7d3-de1c58851333 None admin] PATH_INFO: `/v3/projects/ccf3799e8d9e4dcb8a61bea731cb301d` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.578354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b82fc-f055-4e07-b7d3-de1c58851333 None admin] RBAC: Authorizing `identity:get_project(project_id=ccf3799e8d9e4dcb8a61bea731cb301d)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:15.579786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b82fc-f055-4e07-b7d3-de1c58851333 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:15.581425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 86/176] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:15 2022] GET /identity/v3/projects/ccf3799e8d9e4dcb8a61bea731cb301d => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:15.581891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 91/177] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:15.584902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b82fc-f055-4e07-b7d3-de1c58851333 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:15.585034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8076e3-42d8-4b29-881d-75f193581aab None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.585034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8076e3-42d8-4b29-881d-75f193581aab None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.585034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8076e3-42d8-4b29-881d-75f193581aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.609495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-8d8076e3-42d8-4b29-881d-75f193581aab None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:15.610703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8d8076e3-42d8-4b29-881d-75f193581aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:15.677301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:15.684103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 92/178] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:15.686655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8076e3-42d8-4b29-881d-75f193581aab None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:15.696098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b82fc-f055-4e07-b7d3-de1c58851333 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:15.729350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8592b48-e26f-43d3-88c6-052c1da64945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:15.729980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8592b48-e26f-43d3-88c6-052c1da64945 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.730256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8592b48-e26f-43d3-88c6-052c1da64945 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.730521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8592b48-e26f-43d3-88c6-052c1da64945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.764664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:15.783069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8076e3-42d8-4b29-881d-75f193581aab None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:15.795544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8592b48-e26f-43d3-88c6-052c1da64945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:15.797004 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8592b48-e26f-43d3-88c6-052c1da64945 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:15.816942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80d01d-89ac-4d93-b548-a78676188e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:15.817565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d01d-89ac-4d93-b548-a78676188e2b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.817866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d01d-89ac-4d93-b548-a78676188e2b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.818138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d01d-89ac-4d93-b548-a78676188e2b None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.819254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80d01d-89ac-4d93-b548-a78676188e2b None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:15.820696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80d01d-89ac-4d93-b548-a78676188e2b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:15.822347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 93/179] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:15 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:15.827814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcf91d-948f-4e89-a91f-494c1e309b8c None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.828096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcf91d-948f-4e89-a91f-494c1e309b8c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.828443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcf91d-948f-4e89-a91f-494c1e309b8c None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.829606 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 94/180] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:15.830322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcf91d-948f-4e89-a91f-494c1e309b8c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:15.875798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:15.894552 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 87/181] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:15.895368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8592b48-e26f-43d3-88c6-052c1da64945 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:15.933219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcf91d-948f-4e89-a91f-494c1e309b8c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:15.967626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd52788-7bc1-412a-8bce-f8aedfb4c4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:15.968307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd52788-7bc1-412a-8bce-f8aedfb4c4fa None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:15.968589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd52788-7bc1-412a-8bce-f8aedfb4c4fa None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:15.968860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd52788-7bc1-412a-8bce-f8aedfb4c4fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:15.969581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd52788-7bc1-412a-8bce-f8aedfb4c4fa None admin] RBAC: Adding query filter params (role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=ccf3799e8d9e4dcb8a61bea731cb301d, user.id=3b9f97788d554a69b9c325c4e2da6890) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:15.969919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd52788-7bc1-412a-8bce-f8aedfb4c4fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:15.971360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd52788-7bc1-412a-8bce-f8aedfb4c4fa None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:15.977019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 95/182] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:15 2022] GET /identity/v3/role_assignments?user.id=3b9f97788d554a69b9c325c4e2da6890&scope.project.id=ccf3799e8d9e4dcb8a61bea731cb301d&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:15.977798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd52788-7bc1-412a-8bce-f8aedfb4c4fa None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:15.995384 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:16.000904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8592b48-e26f-43d3-88c6-052c1da64945 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.039823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9ea35-cca2-456e-8091-a471112d9158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.040461 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9ea35-cca2-456e-8091-a471112d9158 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.040735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9ea35-cca2-456e-8091-a471112d9158 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.041004 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9ea35-cca2-456e-8091-a471112d9158 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.052921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9ea35-cca2-456e-8091-a471112d9158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-d9c9ea35-cca2-456e-8091-a471112d9158 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:16.057714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:16.058603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:16.059603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 88/183] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:40:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:16.060245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9ea35-cca2-456e-8091-a471112d9158 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.106130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd52788-7bc1-412a-8bce-f8aedfb4c4fa None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.109869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:16.148431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c35883-1036-4477-8244-fe28596759a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.149068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c35883-1036-4477-8244-fe28596759a4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.149342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c35883-1036-4477-8244-fe28596759a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.149607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c35883-1036-4477-8244-fe28596759a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.162160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c35883-1036-4477-8244-fe28596759a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-a7c35883-1036-4477-8244-fe28596759a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:16.168183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:16.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:16.170583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 96/184] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:16.171683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44deea-e871-4b50-97b3-b4bfd1cd7945 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.171964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44deea-e871-4b50-97b3-b4bfd1cd7945 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.172306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44deea-e871-4b50-97b3-b4bfd1cd7945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.181408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9ea35-cca2-456e-8091-a471112d9158 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.194055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-8b44deea-e871-4b50-97b3-b4bfd1cd7945 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:16.195157 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8b44deea-e871-4b50-97b3-b4bfd1cd7945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:16.221068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eee24b-4bd4-4a81-aaea-7a410eb74d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.221732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eee24b-4bd4-4a81-aaea-7a410eb74d0d None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.222052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eee24b-4bd4-4a81-aaea-7a410eb74d0d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.222322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eee24b-4bd4-4a81-aaea-7a410eb74d0d None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/users/cf123039ca4549b09c08e01db87afcb3/roles/63d2a6efa7784a7eab9594616b3e8978` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.224221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eee24b-4bd4-4a81-aaea-7a410eb74d0d None admin] RBAC: Authorizing `identity:create_grant(project_id=b5bf4929d4984d40a41eaf5bd240ffd0, user_id=cf123039ca4549b09c08e01db87afcb3, role_id=63d2a6efa7784a7eab9594616b3e8978)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:16.225670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eee24b-4bd4-4a81-aaea-7a410eb74d0d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:16.266070 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 97/185] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:16.266991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44deea-e871-4b50-97b3-b4bfd1cd7945 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.269183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:16.306783 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 89/186] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:40:16 2022] PUT /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/users/cf123039ca4549b09c08e01db87afcb3/roles/63d2a6efa7784a7eab9594616b3e8978 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:16.307606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eee24b-4bd4-4a81-aaea-7a410eb74d0d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.376943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44deea-e871-4b50-97b3-b4bfd1cd7945 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.414014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91ca9c-5cf2-464a-b1cc-03e3bb1f1ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.414653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91ca9c-5cf2-464a-b1cc-03e3bb1f1ae1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.415006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91ca9c-5cf2-464a-b1cc-03e3bb1f1ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.415275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91ca9c-5cf2-464a-b1cc-03e3bb1f1ae1 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.416435 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91ca9c-5cf2-464a-b1cc-03e3bb1f1ae1 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:16.417941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91ca9c-5cf2-464a-b1cc-03e3bb1f1ae1 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:16.420796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:16.421431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 98/187] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:16 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:16.422382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91ca9c-5cf2-464a-b1cc-03e3bb1f1ae1 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.429070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eee24b-4bd4-4a81-aaea-7a410eb74d0d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.469761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c95044-084b-48d6-a7f5-8429e9a3edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.470404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95044-084b-48d6-a7f5-8429e9a3edd8 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.470683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95044-084b-48d6-a7f5-8429e9a3edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.470951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95044-084b-48d6-a7f5-8429e9a3edd8 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.471621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c95044-084b-48d6-a7f5-8429e9a3edd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:16.471917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c95044-084b-48d6-a7f5-8429e9a3edd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:16.473286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c95044-084b-48d6-a7f5-8429e9a3edd8 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:16.486001 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 90/188] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:16.486833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c95044-084b-48d6-a7f5-8429e9a3edd8 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.502561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:16.534615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91ca9c-5cf2-464a-b1cc-03e3bb1f1ae1 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.572247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab79104-cac0-4d78-932c-d241d2fe9bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.572959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab79104-cac0-4d78-932c-d241d2fe9bad None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.573240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab79104-cac0-4d78-932c-d241d2fe9bad None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.573512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab79104-cac0-4d78-932c-d241d2fe9bad None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.574215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab79104-cac0-4d78-932c-d241d2fe9bad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:16.574518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab79104-cac0-4d78-932c-d241d2fe9bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:16.575899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab79104-cac0-4d78-932c-d241d2fe9bad None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:16.588997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 99/189] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:16.589924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab79104-cac0-4d78-932c-d241d2fe9bad None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.610189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c95044-084b-48d6-a7f5-8429e9a3edd8 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.645438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923a946-3cd5-496a-8163-3a015051ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.646099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923a946-3cd5-496a-8163-3a015051ef62 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.646382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923a946-3cd5-496a-8163-3a015051ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.646648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923a946-3cd5-496a-8163-3a015051ef62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.647764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923a946-3cd5-496a-8163-3a015051ef62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:16.649206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923a946-3cd5-496a-8163-3a015051ef62 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:16.650811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 91/190] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:16.651572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923a946-3cd5-496a-8163-3a015051ef62 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.692560 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:16.704562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab79104-cac0-4d78-932c-d241d2fe9bad None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.739874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:16.746107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305d96f-d761-449d-b08d-e033d6ce9180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.746743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305d96f-d761-449d-b08d-e033d6ce9180 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.749909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305d96f-d761-449d-b08d-e033d6ce9180 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.749909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305d96f-d761-449d-b08d-e033d6ce9180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.749909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305d96f-d761-449d-b08d-e033d6ce9180 None admin] RBAC: Adding query filter params (role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0, user.id=3b9f97788d554a69b9c325c4e2da6890) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:16.749909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305d96f-d761-449d-b08d-e033d6ce9180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:16.749909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305d96f-d761-449d-b08d-e033d6ce9180 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:16.755934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 100/191] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:16 2022] GET /identity/v3/role_assignments?user.id=3b9f97788d554a69b9c325c4e2da6890&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:16.756719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305d96f-d761-449d-b08d-e033d6ce9180 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.774184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923a946-3cd5-496a-8163-3a015051ef62 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.823804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eabc72-d810-40f6-a26e-5a08d08cd5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.824518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eabc72-d810-40f6-a26e-5a08d08cd5cd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.824840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eabc72-d810-40f6-a26e-5a08d08cd5cd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.825186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eabc72-d810-40f6-a26e-5a08d08cd5cd None admin] PATH_INFO: `/v3/users/30cef38a320f49f7ba5e7b49ecf20109` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.849319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eabc72-d810-40f6-a26e-5a08d08cd5cd None admin] RBAC: Authorizing `identity:get_user(user_id=30cef38a320f49f7ba5e7b49ecf20109)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:16.850841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eabc72-d810-40f6-a26e-5a08d08cd5cd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:16.852581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 92/192] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:16 2022] GET /identity/v3/users/30cef38a320f49f7ba5e7b49ecf20109 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:16.853350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eabc72-d810-40f6-a26e-5a08d08cd5cd None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.892978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305d96f-d761-449d-b08d-e033d6ce9180 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:16.938844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2988c-0777-4e79-9947-320ff3665fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:16.939633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2988c-0777-4e79-9947-320ff3665fc2 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:16.940225 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2988c-0777-4e79-9947-320ff3665fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:16.941129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2988c-0777-4e79-9947-320ff3665fc2 None admin] PATH_INFO: `/v3/users/30cef38a320f49f7ba5e7b49ecf20109` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:16.943882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2988c-0777-4e79-9947-320ff3665fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=30cef38a320f49f7ba5e7b49ecf20109)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:16.943882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2988c-0777-4e79-9947-320ff3665fc2 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:16.945994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 101/193] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:16 2022] GET /identity/v3/users/30cef38a320f49f7ba5e7b49ecf20109 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:16.946154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2988c-0777-4e79-9947-320ff3665fc2 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:16.972954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:16.992022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eabc72-d810-40f6-a26e-5a08d08cd5cd None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:17.018412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:17.037653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3f2cc-4caa-473b-b3ac-70040fb6b5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:17.038353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3f2cc-4caa-473b-b3ac-70040fb6b5b3 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:17.038636 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3f2cc-4caa-473b-b3ac-70040fb6b5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:17.038900 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3f2cc-4caa-473b-b3ac-70040fb6b5b3 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:17.039536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3f2cc-4caa-473b-b3ac-70040fb6b5b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:17.040961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3f2cc-4caa-473b-b3ac-70040fb6b5b3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:17.059060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2988c-0777-4e79-9947-320ff3665fc2 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:17.093382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53868476-b814-4800-bc4a-7b41ca6098d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:17.094013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53868476-b814-4800-bc4a-7b41ca6098d4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:17.094290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53868476-b814-4800-bc4a-7b41ca6098d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:17.094557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53868476-b814-4800-bc4a-7b41ca6098d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:17.159649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 93/194] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:16 2022] POST /identity/v3/users => generated 297 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:17.160448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3f2cc-4caa-473b-b3ac-70040fb6b5b3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:17.169185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53868476-b814-4800-bc4a-7b41ca6098d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:17.172131 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53868476-b814-4800-bc4a-7b41ca6098d4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:17.275557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:17.292838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3f2cc-4caa-473b-b3ac-70040fb6b5b3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:17.335045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 102/195] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:17.340011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e039a18-39e2-4897-b501-e635e3d01151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:17.340678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e039a18-39e2-4897-b501-e635e3d01151 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:17.340881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53868476-b814-4800-bc4a-7b41ca6098d4 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:17.340979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e039a18-39e2-4897-b501-e635e3d01151 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:17.341213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e039a18-39e2-4897-b501-e635e3d01151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:17.361375 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:17.442749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e039a18-39e2-4897-b501-e635e3d01151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:17.444418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e039a18-39e2-4897-b501-e635e3d01151 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:17.480412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53868476-b814-4800-bc4a-7b41ca6098d4 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:17.524672 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c7bea-a3ce-4b9c-bb56-6bb6b88851e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:17.525305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c7bea-a3ce-4b9c-bb56-6bb6b88851e3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:17.525634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c7bea-a3ce-4b9c-bb56-6bb6b88851e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:17.525879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c7bea-a3ce-4b9c-bb56-6bb6b88851e3 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:17.527011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c7bea-a3ce-4b9c-bb56-6bb6b88851e3 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:17.528999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c7bea-a3ce-4b9c-bb56-6bb6b88851e3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:17.531097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 103/196] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:17 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:17.536726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c7bea-a3ce-4b9c-bb56-6bb6b88851e3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:17.572203 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 94/197] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:17.574883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:17.575445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e039a18-39e2-4897-b501-e635e3d01151 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:17.629664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:17.655254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c7bea-a3ce-4b9c-bb56-6bb6b88851e3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:17.687516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e039a18-39e2-4897-b501-e635e3d01151 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:17.690785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba8916-d743-48cd-a16e-f0a6c1c10a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:17.690785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba8916-d743-48cd-a16e-f0a6c1c10a4c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:17.691048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba8916-d743-48cd-a16e-f0a6c1c10a4c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:17.691229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba8916-d743-48cd-a16e-f0a6c1c10a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:17.691956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba8916-d743-48cd-a16e-f0a6c1c10a4c None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=801abb9eafc94aef9a541cb855fb81cd, user.id=30cef38a320f49f7ba5e7b49ecf20109) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:17.692257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba8916-d743-48cd-a16e-f0a6c1c10a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:17.693685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba8916-d743-48cd-a16e-f0a6c1c10a4c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:17.699543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 104/198] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:17 2022] GET /identity/v3/role_assignments?user.id=30cef38a320f49f7ba5e7b49ecf20109&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:17.727307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f49b6-d2b8-47eb-be6e-95383690a49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:17.728223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f49b6-d2b8-47eb-be6e-95383690a49e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:17.728672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f49b6-d2b8-47eb-be6e-95383690a49e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:17.729105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f49b6-d2b8-47eb-be6e-95383690a49e None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:17.730536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f49b6-d2b8-47eb-be6e-95383690a49e None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:17.732730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f49b6-d2b8-47eb-be6e-95383690a49e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:17.734674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 95/199] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:17 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:17.740924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba8916-d743-48cd-a16e-f0a6c1c10a4c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:17.877517 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:17.883616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba8916-d743-48cd-a16e-f0a6c1c10a4c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:17.926539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f31be8-f67a-4090-9be4-c3ba5d209edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:17.929819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f31be8-f67a-4090-9be4-c3ba5d209edc None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:17.929819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f31be8-f67a-4090-9be4-c3ba5d209edc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:17.929819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f31be8-f67a-4090-9be4-c3ba5d209edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:17.929819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f31be8-f67a-4090-9be4-c3ba5d209edc None admin] RBAC: Adding query filter params (role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=801abb9eafc94aef9a541cb855fb81cd, user.id=30cef38a320f49f7ba5e7b49ecf20109) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:17.929819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f31be8-f67a-4090-9be4-c3ba5d209edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:17.930175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f31be8-f67a-4090-9be4-c3ba5d209edc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:17.941984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 105/200] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:17 2022] GET /identity/v3/role_assignments?user.id=30cef38a320f49f7ba5e7b49ecf20109&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:18.490164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c13b94-487a-4ac4-9014-058e1b5d11ca None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:18.490670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c13b94-487a-4ac4-9014-058e1b5d11ca None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:18.490670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c13b94-487a-4ac4-9014-058e1b5d11ca None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:18.492284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 96/201] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:18.497332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b2d1-6803-4b88-92c1-5f2818f4402d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:18.497614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b2d1-6803-4b88-92c1-5f2818f4402d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:18.497920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b2d1-6803-4b88-92c1-5f2818f4402d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:18.516138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ada5b2d1-6803-4b88-92c1-5f2818f4402d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:18.517307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ada5b2d1-6803-4b88-92c1-5f2818f4402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:18.604341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 106/202] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:18.626752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c13b94-487a-4ac4-9014-058e1b5d11ca None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:18.736290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c13b94-487a-4ac4-9014-058e1b5d11ca None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:18.750234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:18.772129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4068e64-059d-4420-9bf9-bc7d32ab4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:18.772759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4068e64-059d-4420-9bf9-bc7d32ab4963 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:18.773030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4068e64-059d-4420-9bf9-bc7d32ab4963 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:18.773295 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4068e64-059d-4420-9bf9-bc7d32ab4963 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:18.784061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4068e64-059d-4420-9bf9-bc7d32ab4963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a4068e64-059d-4420-9bf9-bc7d32ab4963 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:18.788964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:18.789855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:18.791040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 97/203] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:18.796323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5b2d1-6803-4b88-92c1-5f2818f4402d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:18.924428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:18.935066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5b2d1-6803-4b88-92c1-5f2818f4402d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:18.986458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a67421-6564-4c48-8b9e-68d3ab2d751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:18.987069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a67421-6564-4c48-8b9e-68d3ab2d751d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:18.987349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a67421-6564-4c48-8b9e-68d3ab2d751d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:18.987620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a67421-6564-4c48-8b9e-68d3ab2d751d None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:18.988287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a67421-6564-4c48-8b9e-68d3ab2d751d None admin] RBAC: Adding query filter params (name=reader) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:18.988591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a67421-6564-4c48-8b9e-68d3ab2d751d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:18.989995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a67421-6564-4c48-8b9e-68d3ab2d751d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:18.999283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 107/204] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:19.004605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4068e64-059d-4420-9bf9-bc7d32ab4963 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:19.153590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4068e64-059d-4420-9bf9-bc7d32ab4963 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:19.195068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce9f59-6e7e-4ab8-beb3-88842f0a9816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:19.195700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce9f59-6e7e-4ab8-beb3-88842f0a9816 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:19.196032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce9f59-6e7e-4ab8-beb3-88842f0a9816 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:19.196353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce9f59-6e7e-4ab8-beb3-88842f0a9816 None admin] PATH_INFO: `/v3/users/cf123039ca4549b09c08e01db87afcb3` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:19.197524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce9f59-6e7e-4ab8-beb3-88842f0a9816 None admin] RBAC: Authorizing `identity:get_user(user_id=cf123039ca4549b09c08e01db87afcb3)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:19.199087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce9f59-6e7e-4ab8-beb3-88842f0a9816 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:19.201727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 98/205] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:19 2022] GET /identity/v3/users/cf123039ca4549b09c08e01db87afcb3 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:19.207376 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a67421-6564-4c48-8b9e-68d3ab2d751d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:19.323377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a67421-6564-4c48-8b9e-68d3ab2d751d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:19.370278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98486b-e661-485d-bacf-6ea79c9e33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:19.370886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98486b-e661-485d-bacf-6ea79c9e33a5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:19.371165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98486b-e661-485d-bacf-6ea79c9e33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:19.372026 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98486b-e661-485d-bacf-6ea79c9e33a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:19.403246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:19.465234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98486b-e661-485d-bacf-6ea79c9e33a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:19.466746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98486b-e661-485d-bacf-6ea79c9e33a5 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:19.606726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 108/206] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:19.612305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce9f59-6e7e-4ab8-beb3-88842f0a9816 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:19.667884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:19.750712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce9f59-6e7e-4ab8-beb3-88842f0a9816 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:19.784416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f57ba-8b4d-4a72-a4c7-096e00287aa0 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:19.784696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f57ba-8b4d-4a72-a4c7-096e00287aa0 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:19.784963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f57ba-8b4d-4a72-a4c7-096e00287aa0 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:19.786470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 109/207] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:40:19.791404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ca997-dadf-4eee-997f-4f6705fe2b2b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:19.791903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ca997-dadf-4eee-997f-4f6705fe2b2b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:19.792383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ca997-dadf-4eee-997f-4f6705fe2b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:19.807706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d824-1387-40c9-9701-ec80c5a0c793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:19.808500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d824-1387-40c9-9701-ec80c5a0c793 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:19.808944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d824-1387-40c9-9701-ec80c5a0c793 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:19.809221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d824-1387-40c9-9701-ec80c5a0c793 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:19.810399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6d824-1387-40c9-9701-ec80c5a0c793 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:19.811855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6d824-1387-40c9-9701-ec80c5a0c793 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:19.813417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-733ca997-dadf-4eee-997f-4f6705fe2b2b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:19.813853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 99/208] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:19 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:19.815370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-733ca997-dadf-4eee-997f-4f6705fe2b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:19.820525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d824-1387-40c9-9701-ec80c5a0c793 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:19.924118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:19.942057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d824-1387-40c9-9701-ec80c5a0c793 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:19.970158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:19.983181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027adf3-2d51-41d3-9e09-cadeecdcb5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:19.984046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027adf3-2d51-41d3-9e09-cadeecdcb5a4 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:19.984520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027adf3-2d51-41d3-9e09-cadeecdcb5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:19.985960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027adf3-2d51-41d3-9e09-cadeecdcb5a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:19.986908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027adf3-2d51-41d3-9e09-cadeecdcb5a4 None admin] RBAC: Adding query filter params (role.id=63d2a6efa7784a7eab9594616b3e8978, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0, user.id=cf123039ca4549b09c08e01db87afcb3) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:19.988537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027adf3-2d51-41d3-9e09-cadeecdcb5a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:19.991633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027adf3-2d51-41d3-9e09-cadeecdcb5a4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:19.995993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 110/209] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:20.006964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc02c4b-8698-42b5-a668-8cd061794484 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.007246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc02c4b-8698-42b5-a668-8cd061794484 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.007553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc02c4b-8698-42b5-a668-8cd061794484 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.007740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 100/210] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:19 2022] GET /identity/v3/role_assignments?user.id=cf123039ca4549b09c08e01db87afcb3&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=63d2a6efa7784a7eab9594616b3e8978 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:20.010404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 111/211] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:20.017309 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1932d52-51b6-4cf8-971d-15e972640c46 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.017595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1932d52-51b6-4cf8-971d-15e972640c46 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.017949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1932d52-51b6-4cf8-971d-15e972640c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.026643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc02c4b-8698-42b5-a668-8cd061794484 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:20.045140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a1932d52-51b6-4cf8-971d-15e972640c46 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:20.047991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a1932d52-51b6-4cf8-971d-15e972640c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:20.124601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 101/212] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:20.153831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1932d52-51b6-4cf8-971d-15e972640c46 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:20.174233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc02c4b-8698-42b5-a668-8cd061794484 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:20.248940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7b0a9-1a7b-409a-aef3-5b18e01424aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:20.250300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7b0a9-1a7b-409a-aef3-5b18e01424aa None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.251079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7b0a9-1a7b-409a-aef3-5b18e01424aa None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.251519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7b0a9-1a7b-409a-aef3-5b18e01424aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.265117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7b0a9-1a7b-409a-aef3-5b18e01424aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-b9e7b0a9-1a7b-409a-aef3-5b18e01424aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:20.270673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:20.271532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:20.275312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 112/213] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:20.277488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7b0a9-1a7b-409a-aef3-5b18e01424aa None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:20.321728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1932d52-51b6-4cf8-971d-15e972640c46 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:20.325216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:20.334796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:20.362281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4afbe2-95ca-4935-a853-80ba1c5f94be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:20.363161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4afbe2-95ca-4935-a853-80ba1c5f94be None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.363575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4afbe2-95ca-4935-a853-80ba1c5f94be None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.363977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4afbe2-95ca-4935-a853-80ba1c5f94be None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.365439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4afbe2-95ca-4935-a853-80ba1c5f94be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:20.367277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4afbe2-95ca-4935-a853-80ba1c5f94be None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:20.369327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 102/214] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:20.375683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4afbe2-95ca-4935-a853-80ba1c5f94be None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:20.407782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7b0a9-1a7b-409a-aef3-5b18e01424aa None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:20.444405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646e21e-51c1-4330-b756-325c331dd242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:20.444948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646e21e-51c1-4330-b756-325c331dd242 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.445223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646e21e-51c1-4330-b756-325c331dd242 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.445494 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646e21e-51c1-4330-b756-325c331dd242 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.446195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646e21e-51c1-4330-b756-325c331dd242 None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:20.446499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646e21e-51c1-4330-b756-325c331dd242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:20.447911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646e21e-51c1-4330-b756-325c331dd242 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:20.457685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 113/215] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:20.458905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09da228-596d-4084-bbc9-35b1dd9bbd5c None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.459189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09da228-596d-4084-bbc9-35b1dd9bbd5c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.459459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09da228-596d-4084-bbc9-35b1dd9bbd5c None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.460819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 114/216] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:20.465845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09da228-596d-4084-bbc9-35b1dd9bbd5c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:20.502281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4afbe2-95ca-4935-a853-80ba1c5f94be None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:20.537863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfe4d6-d0bf-4703-8e64-61ce29304bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:20.538471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfe4d6-d0bf-4703-8e64-61ce29304bb5 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.538747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfe4d6-d0bf-4703-8e64-61ce29304bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.539012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfe4d6-d0bf-4703-8e64-61ce29304bb5 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.539659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfe4d6-d0bf-4703-8e64-61ce29304bb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:20.541100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfe4d6-d0bf-4703-8e64-61ce29304bb5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:20.571735 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09da228-596d-4084-bbc9-35b1dd9bbd5c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:20.574026 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:20.606205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d4e66-f855-470a-ac19-bec8cd3b525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:20.606831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d4e66-f855-470a-ac19-bec8cd3b525f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.607348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d4e66-f855-470a-ac19-bec8cd3b525f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.607348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d4e66-f855-470a-ac19-bec8cd3b525f None admin] PATH_INFO: `/v3/users/7ead1cdc2cd446e3ad11b8b89423b907` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.620659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d4e66-f855-470a-ac19-bec8cd3b525f None admin] RBAC: Authorizing `identity:get_user(user_id=7ead1cdc2cd446e3ad11b8b89423b907)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:20.622760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d4e66-f855-470a-ac19-bec8cd3b525f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:20.624557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 115/217] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:20 2022] GET /identity/v3/users/7ead1cdc2cd446e3ad11b8b89423b907 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:20.625506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505bb47-7814-4bed-a785-7afeec1f8792 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.625811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505bb47-7814-4bed-a785-7afeec1f8792 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.626073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505bb47-7814-4bed-a785-7afeec1f8792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.643479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1505bb47-7814-4bed-a785-7afeec1f8792 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:20.644649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1505bb47-7814-4bed-a785-7afeec1f8792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:20.644995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 103/218] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:20 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:20.649972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfe4d6-d0bf-4703-8e64-61ce29304bb5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:20.690824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:20.726388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 116/219] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:20.727886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a7c55-26b0-4f89-9019-1a74d9fe57bc None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.728174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a7c55-26b0-4f89-9019-1a74d9fe57bc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.728447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a7c55-26b0-4f89-9019-1a74d9fe57bc None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.730553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 117/220] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:20.735725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154ec3e-5a40-4326-abd9-bee1eaaaeb0b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.736012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154ec3e-5a40-4326-abd9-bee1eaaaeb0b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.736288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154ec3e-5a40-4326-abd9-bee1eaaaeb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.756725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9154ec3e-5a40-4326-abd9-bee1eaaaeb0b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:20.758055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9154ec3e-5a40-4326-abd9-bee1eaaaeb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:20.790128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfe4d6-d0bf-4703-8e64-61ce29304bb5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:20.829996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b4ef6-6955-437a-a3eb-94dcdfff0472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:20.830615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4ef6-6955-437a-a3eb-94dcdfff0472 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:20.830912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4ef6-6955-437a-a3eb-94dcdfff0472 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:20.831167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4ef6-6955-437a-a3eb-94dcdfff0472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:20.837608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 118/221] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:20.838700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154ec3e-5a40-4326-abd9-bee1eaaaeb0b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:20.867304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:20.926087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b4ef6-6955-437a-a3eb-94dcdfff0472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:20.927580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b4ef6-6955-437a-a3eb-94dcdfff0472 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:20.962412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:20.967975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154ec3e-5a40-4326-abd9-bee1eaaaeb0b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.014028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecdb37-1a5b-46f7-b1af-e02fac46cb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:21.014588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecdb37-1a5b-46f7-b1af-e02fac46cb27 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:21.014867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecdb37-1a5b-46f7-b1af-e02fac46cb27 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:21.015133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecdb37-1a5b-46f7-b1af-e02fac46cb27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:21.029684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecdb37-1a5b-46f7-b1af-e02fac46cb27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-e9ecdb37-1a5b-46f7-b1af-e02fac46cb27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:21.035352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:21.036415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 119/222] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:40:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:21.037330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecdb37-1a5b-46f7-b1af-e02fac46cb27 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.049505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 104/223] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:21.050570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b4ef6-6955-437a-a3eb-94dcdfff0472 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.132045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:21.151329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecdb37-1a5b-46f7-b1af-e02fac46cb27 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.189317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12da4b-b0f1-4593-af80-36ba801b9e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:21.190307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12da4b-b0f1-4593-af80-36ba801b9e8a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:21.190548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b4ef6-6955-437a-a3eb-94dcdfff0472 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.191101 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12da4b-b0f1-4593-af80-36ba801b9e8a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:21.191691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12da4b-b0f1-4593-af80-36ba801b9e8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:21.203344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12da4b-b0f1-4593-af80-36ba801b9e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-1f12da4b-b0f1-4593-af80-36ba801b9e8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:21.208269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:21.209375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:21.210291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:21.217416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 120/224] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:21.218488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12da4b-b0f1-4593-af80-36ba801b9e8a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.220566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:21.232072 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3838d0-e5d5-4351-a5c3-d7b0c9e56eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:21.233169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3838d0-e5d5-4351-a5c3-d7b0c9e56eb7 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:21.233453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3838d0-e5d5-4351-a5c3-d7b0c9e56eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:21.233729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3838d0-e5d5-4351-a5c3-d7b0c9e56eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:21.234853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3838d0-e5d5-4351-a5c3-d7b0c9e56eb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:21.235958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3838d0-e5d5-4351-a5c3-d7b0c9e56eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:21.236977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3838d0-e5d5-4351-a5c3-d7b0c9e56eb7 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:21.249316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 105/225] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:21.250187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3838d0-e5d5-4351-a5c3-d7b0c9e56eb7 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.352215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12da4b-b0f1-4593-af80-36ba801b9e8a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.361420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3838d0-e5d5-4351-a5c3-d7b0c9e56eb7 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.396087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c001e-51c6-43f6-bd09-d70cff465529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:21.396287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6c21b-ebef-4c05-a737-d163561bcb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:21.396693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c001e-51c6-43f6-bd09-d70cff465529 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:21.396961 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6c21b-ebef-4c05-a737-d163561bcb57 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:21.397061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c001e-51c6-43f6-bd09-d70cff465529 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:21.397227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c001e-51c6-43f6-bd09-d70cff465529 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:21.397319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6c21b-ebef-4c05-a737-d163561bcb57 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:21.397683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6c21b-ebef-4c05-a737-d163561bcb57 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:21.397909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c001e-51c6-43f6-bd09-d70cff465529 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:21.398208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c001e-51c6-43f6-bd09-d70cff465529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:21.399056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6c21b-ebef-4c05-a737-d163561bcb57 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:21.399681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c001e-51c6-43f6-bd09-d70cff465529 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:21.400665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6c21b-ebef-4c05-a737-d163561bcb57 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:21.403287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 121/226] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:21 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:21.406663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6c21b-ebef-4c05-a737-d163561bcb57 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.419269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 106/227] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:21.420170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c001e-51c6-43f6-bd09-d70cff465529 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.421106 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:21.485664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:21.529333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6c21b-ebef-4c05-a737-d163561bcb57 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.540247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c001e-51c6-43f6-bd09-d70cff465529 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.573604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b2190-a5f8-482d-b949-6accdb99b53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:21.574564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b2190-a5f8-482d-b949-6accdb99b53e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:21.574990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b2190-a5f8-482d-b949-6accdb99b53e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:21.575429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b2190-a5f8-482d-b949-6accdb99b53e None admin] PATH_INFO: `/v3/users/30cef38a320f49f7ba5e7b49ecf20109` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:21.576593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3589d0-e3d5-47e2-a66d-067286d9eac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:21.577118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b2190-a5f8-482d-b949-6accdb99b53e None admin] RBAC: Authorizing `identity:get_user(user_id=30cef38a320f49f7ba5e7b49ecf20109)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:21.577232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3589d0-e3d5-47e2-a66d-067286d9eac4 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:21.577452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3589d0-e3d5-47e2-a66d-067286d9eac4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:21.577716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3589d0-e3d5-47e2-a66d-067286d9eac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:21.578593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3589d0-e3d5-47e2-a66d-067286d9eac4 None admin] RBAC: Adding query filter params (role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=801abb9eafc94aef9a541cb855fb81cd, user.id=7ead1cdc2cd446e3ad11b8b89423b907) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:21.578900 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3589d0-e3d5-47e2-a66d-067286d9eac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:21.579470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b2190-a5f8-482d-b949-6accdb99b53e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:21.580326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3589d0-e3d5-47e2-a66d-067286d9eac4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:21.581397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 122/228] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:21 2022] GET /identity/v3/users/30cef38a320f49f7ba5e7b49ecf20109 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:21.582450 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b2190-a5f8-482d-b949-6accdb99b53e None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.596750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 107/229] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:21 2022] GET /identity/v3/role_assignments?user.id=7ead1cdc2cd446e3ad11b8b89423b907&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:21.596750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3589d0-e3d5-47e2-a66d-067286d9eac4 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.733172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:21.778591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b2190-a5f8-482d-b949-6accdb99b53e None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.810273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3589d0-e3d5-47e2-a66d-067286d9eac4 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.827974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:21.837075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c65d7-1ecc-4a53-ad7f-656f4807eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:21.837949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c65d7-1ecc-4a53-ad7f-656f4807eadb None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:21.838398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c65d7-1ecc-4a53-ad7f-656f4807eadb None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:21.838813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c65d7-1ecc-4a53-ad7f-656f4807eadb None admin] PATH_INFO: `/v3/users/30cef38a320f49f7ba5e7b49ecf20109` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:21.840683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c65d7-1ecc-4a53-ad7f-656f4807eadb None admin] RBAC: Authorizing `identity:get_user(user_id=30cef38a320f49f7ba5e7b49ecf20109)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:21.842407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c65d7-1ecc-4a53-ad7f-656f4807eadb None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:21.844418 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 123/230] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:21 2022] GET /identity/v3/users/30cef38a320f49f7ba5e7b49ecf20109 => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:21.852188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c65d7-1ecc-4a53-ad7f-656f4807eadb None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.855994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fe349-f0a5-409b-a841-5a35b3046321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:21.857402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fe349-f0a5-409b-a841-5a35b3046321 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:21.857402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fe349-f0a5-409b-a841-5a35b3046321 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:21.857402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fe349-f0a5-409b-a841-5a35b3046321 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:21.859607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fe349-f0a5-409b-a841-5a35b3046321 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:21.861187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fe349-f0a5-409b-a841-5a35b3046321 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:21.867615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 108/231] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:21 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:21.873237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fe349-f0a5-409b-a841-5a35b3046321 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:21.967539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c65d7-1ecc-4a53-ad7f-656f4807eadb None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:21.982081 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fe349-f0a5-409b-a841-5a35b3046321 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:22.019430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcfd34-201e-4ea8-a966-35ea8ea3458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:22.019996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcfd34-201e-4ea8-a966-35ea8ea3458a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:22.020273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcfd34-201e-4ea8-a966-35ea8ea3458a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:22.021179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcfd34-201e-4ea8-a966-35ea8ea3458a None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:22.022362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d293076-2d65-4852-865e-f8e91dcb40e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:22.023039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d293076-2d65-4852-865e-f8e91dcb40e4 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:22.023310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d293076-2d65-4852-865e-f8e91dcb40e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:22.023580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d293076-2d65-4852-865e-f8e91dcb40e4 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd/users/30cef38a320f49f7ba5e7b49ecf20109/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:22.025057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcfd34-201e-4ea8-a966-35ea8ea3458a None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:22.027011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcfd34-201e-4ea8-a966-35ea8ea3458a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:22.029163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d293076-2d65-4852-865e-f8e91dcb40e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=801abb9eafc94aef9a541cb855fb81cd, user_id=30cef38a320f49f7ba5e7b49ecf20109, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:22.031624 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 124/232] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:21 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:22.031808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d293076-2d65-4852-865e-f8e91dcb40e4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:22.041976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcfd34-201e-4ea8-a966-35ea8ea3458a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:22.060125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:22.113553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 109/233] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:40:21 2022] PUT /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd/users/30cef38a320f49f7ba5e7b49ecf20109/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:40:22.164990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcfd34-201e-4ea8-a966-35ea8ea3458a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:22.217554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc788491-f6ac-440b-8a94-fae375c79d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:22.218312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788491-f6ac-440b-8a94-fae375c79d96 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:22.218596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788491-f6ac-440b-8a94-fae375c79d96 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:22.218865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788491-f6ac-440b-8a94-fae375c79d96 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd/users/30cef38a320f49f7ba5e7b49ecf20109/roles/d7d75f4c26eb44689ec71efb8b63e534` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:22.221187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc788491-f6ac-440b-8a94-fae375c79d96 None admin] RBAC: Authorizing `identity:create_grant(project_id=801abb9eafc94aef9a541cb855fb81cd, user_id=30cef38a320f49f7ba5e7b49ecf20109, role_id=d7d75f4c26eb44689ec71efb8b63e534)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:22.222685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc788491-f6ac-440b-8a94-fae375c79d96 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:22.297673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 125/234] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:40:22 2022] PUT /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd/users/30cef38a320f49f7ba5e7b49ecf20109/roles/d7d75f4c26eb44689ec71efb8b63e534 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:40:23.035107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e208b-a19e-476c-91db-882f6170da96 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.036173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e208b-a19e-476c-91db-882f6170da96 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.036652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e208b-a19e-476c-91db-882f6170da96 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.038266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 110/235] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:23.044310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede555e3-4889-40a3-80d5-2b51c0f7725b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.044742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede555e3-4889-40a3-80d5-2b51c0f7725b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.044917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede555e3-4889-40a3-80d5-2b51c0f7725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.066943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ede555e3-4889-40a3-80d5-2b51c0f7725b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:23.068116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ede555e3-4889-40a3-80d5-2b51c0f7725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:23.123446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66762b43-b946-44e9-9d43-3d9013d17ba6 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.124051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66762b43-b946-44e9-9d43-3d9013d17ba6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.124734 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66762b43-b946-44e9-9d43-3d9013d17ba6 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.127366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 111/236] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:23.133155 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb94e4-e1da-4fd3-a6bb-cee3480cf852 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.134285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb94e4-e1da-4fd3-a6bb-cee3480cf852 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.134765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb94e4-e1da-4fd3-a6bb-cee3480cf852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.152349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 126/237] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:23.156157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-dcbb94e4-e1da-4fd3-a6bb-cee3480cf852 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:23.157570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-dcbb94e4-e1da-4fd3-a6bb-cee3480cf852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:23.193177 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:23.194669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede555e3-4889-40a3-80d5-2b51c0f7725b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:23.202045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:23.252580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 112/238] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:23.276182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb94e4-e1da-4fd3-a6bb-cee3480cf852 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:23.438491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede555e3-4889-40a3-80d5-2b51c0f7725b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:23.469358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb94e4-e1da-4fd3-a6bb-cee3480cf852 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:23.512386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb7d07-fc95-40d6-9a28-d50ee1dbce0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:23.514047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb7d07-fc95-40d6-9a28-d50ee1dbce0e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.514334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb7d07-fc95-40d6-9a28-d50ee1dbce0e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.514608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb7d07-fc95-40d6-9a28-d50ee1dbce0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.516649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d6c36-868a-4a43-8ab3-5dd2d87c6f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:23.517271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d6c36-868a-4a43-8ab3-5dd2d87c6f87 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.517538 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d6c36-868a-4a43-8ab3-5dd2d87c6f87 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.517813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d6c36-868a-4a43-8ab3-5dd2d87c6f87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.521104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d6c36-868a-4a43-8ab3-5dd2d87c6f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:23.524197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d6c36-868a-4a43-8ab3-5dd2d87c6f87 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:23.525776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 127/239] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:23.531675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d6c36-868a-4a43-8ab3-5dd2d87c6f87 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:23.540197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb7d07-fc95-40d6-9a28-d50ee1dbce0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-2dcb7d07-fc95-40d6-9a28-d50ee1dbce0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:23.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:23.547250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:23.548229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 113/240] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:23.555492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb7d07-fc95-40d6-9a28-d50ee1dbce0e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:23.604973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:23.632669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:23.662826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d6c36-868a-4a43-8ab3-5dd2d87c6f87 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:23.697433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc211fb-622d-40d4-8f46-495cb8737ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:23.703877 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc211fb-622d-40d4-8f46-495cb8737ab3 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.705868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc211fb-622d-40d4-8f46-495cb8737ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.706145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc211fb-622d-40d4-8f46-495cb8737ab3 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.706805 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc211fb-622d-40d4-8f46-495cb8737ab3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:23.709382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc211fb-622d-40d4-8f46-495cb8737ab3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:23.720915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb7d07-fc95-40d6-9a28-d50ee1dbce0e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:23.761302 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a6ec8-1b1f-4725-887a-f0d92715ed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:23.762218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a6ec8-1b1f-4725-887a-f0d92715ed9d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.762648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a6ec8-1b1f-4725-887a-f0d92715ed9d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.763100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a6ec8-1b1f-4725-887a-f0d92715ed9d None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.764063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a6ec8-1b1f-4725-887a-f0d92715ed9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:23.764546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a6ec8-1b1f-4725-887a-f0d92715ed9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:23.766458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a6ec8-1b1f-4725-887a-f0d92715ed9d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:23.775990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 114/241] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:23.782139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a6ec8-1b1f-4725-887a-f0d92715ed9d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:23.819285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 128/242] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:23 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:40:23.892578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:23.900512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a6ec8-1b1f-4725-887a-f0d92715ed9d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:23.940315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71dfea-cfb0-4610-961e-e5975ce34afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:23.941292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71dfea-cfb0-4610-961e-e5975ce34afb None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.941714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71dfea-cfb0-4610-961e-e5975ce34afb None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.942200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71dfea-cfb0-4610-961e-e5975ce34afb None admin] PATH_INFO: `/v3/users/d11f4676e1a24dae86e0730e6625beaf` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.959039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71dfea-cfb0-4610-961e-e5975ce34afb None admin] RBAC: Authorizing `identity:get_user(user_id=d11f4676e1a24dae86e0730e6625beaf)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:23.960756 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71dfea-cfb0-4610-961e-e5975ce34afb None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:23.962684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 115/243] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:23 2022] GET /identity/v3/users/d11f4676e1a24dae86e0730e6625beaf => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:23.965823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b392cef-3b26-4736-9f7b-be9fd97c20ac None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.965823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b392cef-3b26-4736-9f7b-be9fd97c20ac None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.965823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b392cef-3b26-4736-9f7b-be9fd97c20ac None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:23.969193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71dfea-cfb0-4610-961e-e5975ce34afb None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:23.970365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 129/244] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:23.976239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdae24-485a-4f8d-8b09-2f3aaac6fb3c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:23.976648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdae24-485a-4f8d-8b09-2f3aaac6fb3c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:23.977013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdae24-485a-4f8d-8b09-2f3aaac6fb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:24.010353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b4cdae24-485a-4f8d-8b09-2f3aaac6fb3c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:24.011705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b4cdae24-485a-4f8d-8b09-2f3aaac6fb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:24.077287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71dfea-cfb0-4610-961e-e5975ce34afb None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:24.113167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91a422-65fb-4117-8b7f-b22f238f43ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:24.113780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91a422-65fb-4117-8b7f-b22f238f43ea None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:24.114083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91a422-65fb-4117-8b7f-b22f238f43ea None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:24.114348 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91a422-65fb-4117-8b7f-b22f238f43ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:24.138719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:24.140469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 130/245] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:24.165111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdae24-485a-4f8d-8b09-2f3aaac6fb3c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:24.189665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91a422-65fb-4117-8b7f-b22f238f43ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:24.191202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91a422-65fb-4117-8b7f-b22f238f43ea None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:24.260967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:24.321823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdae24-485a-4f8d-8b09-2f3aaac6fb3c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:24.345999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 116/246] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:24.349440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91a422-65fb-4117-8b7f-b22f238f43ea None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:24.377220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1185e-07fd-4aca-bf8b-8dfb0902c312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:24.378285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1185e-07fd-4aca-bf8b-8dfb0902c312 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:24.378770 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1185e-07fd-4aca-bf8b-8dfb0902c312 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:24.379236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1185e-07fd-4aca-bf8b-8dfb0902c312 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:24.394762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1185e-07fd-4aca-bf8b-8dfb0902c312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-f7c1185e-07fd-4aca-bf8b-8dfb0902c312 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:24.400031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:24.400901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:24.401679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:24.409887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 131/247] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:24.411883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1185e-07fd-4aca-bf8b-8dfb0902c312 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:24.415030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:24.476948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91a422-65fb-4117-8b7f-b22f238f43ea None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:24.520990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd53c4d-907d-44d2-b656-26ff779fc3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:24.521569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd53c4d-907d-44d2-b656-26ff779fc3c2 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:24.522145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd53c4d-907d-44d2-b656-26ff779fc3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:24.522424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd53c4d-907d-44d2-b656-26ff779fc3c2 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:24.524371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd53c4d-907d-44d2-b656-26ff779fc3c2 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:24.525876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd53c4d-907d-44d2-b656-26ff779fc3c2 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:24.528032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 117/248] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:24 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:24.536906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1185e-07fd-4aca-bf8b-8dfb0902c312 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:24.538923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd53c4d-907d-44d2-b656-26ff779fc3c2 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:24.557445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:24.607489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85418f1f-a90f-4aa7-af95-5372f4bf1a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:24.608112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85418f1f-a90f-4aa7-af95-5372f4bf1a08 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:24.608391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85418f1f-a90f-4aa7-af95-5372f4bf1a08 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:24.608659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85418f1f-a90f-4aa7-af95-5372f4bf1a08 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:24.609336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85418f1f-a90f-4aa7-af95-5372f4bf1a08 None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:24.609641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85418f1f-a90f-4aa7-af95-5372f4bf1a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:24.619264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85418f1f-a90f-4aa7-af95-5372f4bf1a08 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:24.647207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 132/249] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:24.652366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85418f1f-a90f-4aa7-af95-5372f4bf1a08 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:24.796705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd53c4d-907d-44d2-b656-26ff779fc3c2 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:24.839312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:24.857714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85418f1f-a90f-4aa7-af95-5372f4bf1a08 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:24.871986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444efd6-ca52-424d-befa-c828fd46f8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:24.872842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444efd6-ca52-424d-befa-c828fd46f8e5 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:24.873273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444efd6-ca52-424d-befa-c828fd46f8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:24.873722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444efd6-ca52-424d-befa-c828fd46f8e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:24.878699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444efd6-ca52-424d-befa-c828fd46f8e5 None admin] RBAC: Adding query filter params (role.id=63d2a6efa7784a7eab9594616b3e8978, scope.project.id=801abb9eafc94aef9a541cb855fb81cd, user.id=d11f4676e1a24dae86e0730e6625beaf) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:24.879240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444efd6-ca52-424d-befa-c828fd46f8e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:24.880844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444efd6-ca52-424d-befa-c828fd46f8e5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:24.890422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 118/250] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:24 2022] GET /identity/v3/role_assignments?user.id=d11f4676e1a24dae86e0730e6625beaf&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=63d2a6efa7784a7eab9594616b3e8978 => generated 259 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:24.932816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1112d4e-598d-4f60-ba3c-3b0eb658fc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:24.933503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1112d4e-598d-4f60-ba3c-3b0eb658fc01 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:24.934429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1112d4e-598d-4f60-ba3c-3b0eb658fc01 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:24.934694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1112d4e-598d-4f60-ba3c-3b0eb658fc01 None admin] PATH_INFO: `/v3/users/7ead1cdc2cd446e3ad11b8b89423b907` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:24.935957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1112d4e-598d-4f60-ba3c-3b0eb658fc01 None admin] RBAC: Authorizing `identity:get_user(user_id=7ead1cdc2cd446e3ad11b8b89423b907)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:24.937439 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1112d4e-598d-4f60-ba3c-3b0eb658fc01 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:24.939117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 133/251] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:24 2022] GET /identity/v3/users/7ead1cdc2cd446e3ad11b8b89423b907 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:24.945172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444efd6-ca52-424d-befa-c828fd46f8e5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:25.025299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fdd1c-f68c-4f13-b59a-25e8afc97db7 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.025572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fdd1c-f68c-4f13-b59a-25e8afc97db7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.025933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fdd1c-f68c-4f13-b59a-25e8afc97db7 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.027999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 134/252] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:25.035165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf40092-0b82-49a1-bd6b-c2a5f749efd5 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.035463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf40092-0b82-49a1-bd6b-c2a5f749efd5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.035704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf40092-0b82-49a1-bd6b-c2a5f749efd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.070818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-8bf40092-0b82-49a1-bd6b-c2a5f749efd5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:25.072377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8bf40092-0b82-49a1-bd6b-c2a5f749efd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:25.126046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444efd6-ca52-424d-befa-c828fd46f8e5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:25.166452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:25.167281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4695b2-bc48-488a-9fbf-bfd36d2e096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:25.168779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4695b2-bc48-488a-9fbf-bfd36d2e096d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.169057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4695b2-bc48-488a-9fbf-bfd36d2e096d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.169329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4695b2-bc48-488a-9fbf-bfd36d2e096d None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.170693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4695b2-bc48-488a-9fbf-bfd36d2e096d None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:25.172164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4695b2-bc48-488a-9fbf-bfd36d2e096d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:25.173852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 119/253] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:24 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:25.182752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4695b2-bc48-488a-9fbf-bfd36d2e096d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:25.204746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 135/254] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:25.205890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70613ea7-d965-4ef8-9fef-026d7b5277d3 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.206166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70613ea7-d965-4ef8-9fef-026d7b5277d3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.206432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70613ea7-d965-4ef8-9fef-026d7b5277d3 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.207800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 136/255] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:25.213485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047f5b-a202-409b-ac36-aca9e09b8a37 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.214049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047f5b-a202-409b-ac36-aca9e09b8a37 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.214524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047f5b-a202-409b-ac36-aca9e09b8a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.234652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-30047f5b-a202-409b-ac36-aca9e09b8a37 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:25.236168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-30047f5b-a202-409b-ac36-aca9e09b8a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:25.254557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:25.311385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 137/256] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:25.318048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4695b2-bc48-488a-9fbf-bfd36d2e096d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:25.319222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047f5b-a202-409b-ac36-aca9e09b8a37 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:25.358667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bed1a-c44e-4479-9391-687774e88750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:25.359229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bed1a-c44e-4479-9391-687774e88750 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.359502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bed1a-c44e-4479-9391-687774e88750 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.359767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bed1a-c44e-4479-9391-687774e88750 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd/users/7ead1cdc2cd446e3ad11b8b89423b907/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.363596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bed1a-c44e-4479-9391-687774e88750 None admin] RBAC: Authorizing `identity:create_grant(project_id=801abb9eafc94aef9a541cb855fb81cd, user_id=7ead1cdc2cd446e3ad11b8b89423b907, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:25.365200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bed1a-c44e-4479-9391-687774e88750 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:25.438172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 120/257] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:40:25 2022] PUT /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd/users/7ead1cdc2cd446e3ad11b8b89423b907/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:25.438788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bed1a-c44e-4479-9391-687774e88750 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:25.483722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047f5b-a202-409b-ac36-aca9e09b8a37 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:25.501607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:25.555646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72732739-11ea-4937-abbe-26f43b272192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:25.556282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72732739-11ea-4937-abbe-26f43b272192 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.556555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72732739-11ea-4937-abbe-26f43b272192 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.556817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72732739-11ea-4937-abbe-26f43b272192 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.580056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72732739-11ea-4937-abbe-26f43b272192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-72732739-11ea-4937-abbe-26f43b272192 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:25.584748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:25.585592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:25.586412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:25.587566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 138/258] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:25.602847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72732739-11ea-4937-abbe-26f43b272192 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:25.630406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bed1a-c44e-4479-9391-687774e88750 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:25.644500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:25.674415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3670315-32d4-4734-8a3b-0160a5aeff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:25.675071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3670315-32d4-4734-8a3b-0160a5aeff3d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.675405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3670315-32d4-4734-8a3b-0160a5aeff3d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.675673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3670315-32d4-4734-8a3b-0160a5aeff3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.695048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3670315-32d4-4734-8a3b-0160a5aeff3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-e3670315-32d4-4734-8a3b-0160a5aeff3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:25.699934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:25.701114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:25.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 121/259] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:40:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:25.715467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3670315-32d4-4734-8a3b-0160a5aeff3d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:25.753127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72732739-11ea-4937-abbe-26f43b272192 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:25.821828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cdd63-ece3-4c5c-a060-cb1f0f3f708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:25.822445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cdd63-ece3-4c5c-a060-cb1f0f3f708b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.822717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cdd63-ece3-4c5c-a060-cb1f0f3f708b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.822980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cdd63-ece3-4c5c-a060-cb1f0f3f708b None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.823649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cdd63-ece3-4c5c-a060-cb1f0f3f708b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:25.823942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cdd63-ece3-4c5c-a060-cb1f0f3f708b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:25.826165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cdd63-ece3-4c5c-a060-cb1f0f3f708b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:25.841246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 139/260] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:25.847349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cdd63-ece3-4c5c-a060-cb1f0f3f708b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:25.878300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3670315-32d4-4734-8a3b-0160a5aeff3d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:25.897848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:25.931559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490e9b7-8984-4e0d-bb31-25d175e7b999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:25.932449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490e9b7-8984-4e0d-bb31-25d175e7b999 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:25.932733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490e9b7-8984-4e0d-bb31-25d175e7b999 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:25.933005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490e9b7-8984-4e0d-bb31-25d175e7b999 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:25.934217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490e9b7-8984-4e0d-bb31-25d175e7b999 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:25.934517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490e9b7-8984-4e0d-bb31-25d175e7b999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:25.936148 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490e9b7-8984-4e0d-bb31-25d175e7b999 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:25.951885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 122/261] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:25.965724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490e9b7-8984-4e0d-bb31-25d175e7b999 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:26.040886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:26.047880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cdd63-ece3-4c5c-a060-cb1f0f3f708b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:26.138309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa2ea4-f968-467f-a20d-88497d4a6db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:26.139241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa2ea4-f968-467f-a20d-88497d4a6db3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:26.139715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa2ea4-f968-467f-a20d-88497d4a6db3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:26.140154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa2ea4-f968-467f-a20d-88497d4a6db3 None admin] PATH_INFO: `/v3/users/30cef38a320f49f7ba5e7b49ecf20109` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:26.146462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa2ea4-f968-467f-a20d-88497d4a6db3 None admin] RBAC: Authorizing `identity:get_user(user_id=30cef38a320f49f7ba5e7b49ecf20109)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:26.148220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa2ea4-f968-467f-a20d-88497d4a6db3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:26.150375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 140/262] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:25 2022] GET /identity/v3/users/30cef38a320f49f7ba5e7b49ecf20109 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:26.156966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa2ea4-f968-467f-a20d-88497d4a6db3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:26.171083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490e9b7-8984-4e0d-bb31-25d175e7b999 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:26.251292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ff20-0b94-43d3-8b37-23d846257351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:26.251976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ff20-0b94-43d3-8b37-23d846257351 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:26.252256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ff20-0b94-43d3-8b37-23d846257351 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:26.252524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ff20-0b94-43d3-8b37-23d846257351 None admin] PATH_INFO: `/v3/users/30cef38a320f49f7ba5e7b49ecf20109` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:26.258492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ff20-0b94-43d3-8b37-23d846257351 None admin] RBAC: Authorizing `identity:get_user(user_id=30cef38a320f49f7ba5e7b49ecf20109)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:26.259969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ff20-0b94-43d3-8b37-23d846257351 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:26.261880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 123/263] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:25 2022] GET /identity/v3/users/30cef38a320f49f7ba5e7b49ecf20109 => generated 283 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:26.271872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ff20-0b94-43d3-8b37-23d846257351 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:26.367465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:26.374812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa2ea4-f968-467f-a20d-88497d4a6db3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:26.434547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b04fcf-b690-4e75-9db5-ffd0d148c372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:26.435128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b04fcf-b690-4e75-9db5-ffd0d148c372 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:26.435398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b04fcf-b690-4e75-9db5-ffd0d148c372 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:26.435665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b04fcf-b690-4e75-9db5-ffd0d148c372 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:26.483241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ff20-0b94-43d3-8b37-23d846257351 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:26.485887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:26.547713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d05127-40fd-4ccd-a6b4-eb32b142560d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:26.548313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d05127-40fd-4ccd-a6b4-eb32b142560d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:26.548581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d05127-40fd-4ccd-a6b4-eb32b142560d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:26.551254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d05127-40fd-4ccd-a6b4-eb32b142560d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:26.559145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b04fcf-b690-4e75-9db5-ffd0d148c372 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:26.560666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b04fcf-b690-4e75-9db5-ffd0d148c372 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:26.679251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d05127-40fd-4ccd-a6b4-eb32b142560d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:26.681111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d05127-40fd-4ccd-a6b4-eb32b142560d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:26.839155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:26.849191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 141/264] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:26.856074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b04fcf-b690-4e75-9db5-ffd0d148c372 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:26.864118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 124/265] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 16:40:26.869491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d05127-40fd-4ccd-a6b4-eb32b142560d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:26.872246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:27.043537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b04fcf-b690-4e75-9db5-ffd0d148c372 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:27.048932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d05127-40fd-4ccd-a6b4-eb32b142560d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:27.092627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b9eb8-61e4-492a-8832-b7aa27183f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:27.092819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb3766-3a51-46a8-a0f8-abf251765833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:27.093212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb3766-3a51-46a8-a0f8-abf251765833 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.093495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb3766-3a51-46a8-a0f8-abf251765833 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.093766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb3766-3a51-46a8-a0f8-abf251765833 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.094194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b9eb8-61e4-492a-8832-b7aa27183f3a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.094606 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b9eb8-61e4-492a-8832-b7aa27183f3a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.095027 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b9eb8-61e4-492a-8832-b7aa27183f3a None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.095132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb3766-3a51-46a8-a0f8-abf251765833 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:27.096356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b9eb8-61e4-492a-8832-b7aa27183f3a None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:27.096616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb3766-3a51-46a8-a0f8-abf251765833 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:27.098082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b9eb8-61e4-492a-8832-b7aa27183f3a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:27.098303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 125/266] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:26 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:27.100242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 142/267] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:26 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:27.101496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc7947-3219-4297-861e-e170b5bda8fd None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.101935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc7947-3219-4297-861e-e170b5bda8fd None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.102338 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc7947-3219-4297-861e-e170b5bda8fd None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.103905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 143/268] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:27.108326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec5f86-6278-4a12-adcc-d38be98017ef None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.108326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec5f86-6278-4a12-adcc-d38be98017ef None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.108326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec5f86-6278-4a12-adcc-d38be98017ef None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.109844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc7947-3219-4297-861e-e170b5bda8fd None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:27.110354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 126/269] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:27.113035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6c49-f94e-43a0-b775-a7d8d7ce10b6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.113320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6c49-f94e-43a0-b775-a7d8d7ce10b6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.113654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6c49-f94e-43a0-b775-a7d8d7ce10b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.136156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f9da6c49-f94e-43a0-b775-a7d8d7ce10b6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:27.137348 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f9da6c49-f94e-43a0-b775-a7d8d7ce10b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:27.187194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:27.231899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:27.232414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc7947-3219-4297-861e-e170b5bda8fd None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:27.238049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 127/270] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:27.246887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da6c49-f94e-43a0-b775-a7d8d7ce10b6 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:27.271721 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54414b-03f2-4955-b458-336d75b96bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:27.272326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54414b-03f2-4955-b458-336d75b96bce None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.272630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54414b-03f2-4955-b458-336d75b96bce None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.273182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54414b-03f2-4955-b458-336d75b96bce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.274021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54414b-03f2-4955-b458-336d75b96bce None admin] RBAC: Adding query filter params (role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=801abb9eafc94aef9a541cb855fb81cd, user.id=30cef38a320f49f7ba5e7b49ecf20109) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:27.274335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54414b-03f2-4955-b458-336d75b96bce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:27.276191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54414b-03f2-4955-b458-336d75b96bce None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:27.283420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 144/271] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:27 2022] GET /identity/v3/role_assignments?user.id=30cef38a320f49f7ba5e7b49ecf20109&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:27.284593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949aecb-b040-4319-850b-58baa23133ae None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.284872 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949aecb-b040-4319-850b-58baa23133ae None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.290858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949aecb-b040-4319-850b-58baa23133ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.310596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a949aecb-b040-4319-850b-58baa23133ae None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:27.312100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a949aecb-b040-4319-850b-58baa23133ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:27.375581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da6c49-f94e-43a0-b775-a7d8d7ce10b6 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:27.408762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 145/272] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:27.409638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949aecb-b040-4319-850b-58baa23133ae None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:27.414338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bc0e2-977a-422c-8084-59afc87b9f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:27.415332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bc0e2-977a-422c-8084-59afc87b9f1c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.415833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bc0e2-977a-422c-8084-59afc87b9f1c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.416291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bc0e2-977a-422c-8084-59afc87b9f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.417285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bc0e2-977a-422c-8084-59afc87b9f1c None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=801abb9eafc94aef9a541cb855fb81cd, user.id=30cef38a320f49f7ba5e7b49ecf20109) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:27.418979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bc0e2-977a-422c-8084-59afc87b9f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:27.420671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bc0e2-977a-422c-8084-59afc87b9f1c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:27.427334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 128/273] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:27 2022] GET /identity/v3/role_assignments?user.id=30cef38a320f49f7ba5e7b49ecf20109&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:27.439933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bc0e2-977a-422c-8084-59afc87b9f1c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:27.493457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:27.539114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:27.576853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949aecb-b040-4319-850b-58baa23133ae None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:27.604927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bc0e2-977a-422c-8084-59afc87b9f1c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:27.644384 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744af80f-b917-4257-be28-fb1d308dcf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:27.644955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744af80f-b917-4257-be28-fb1d308dcf40 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.645233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744af80f-b917-4257-be28-fb1d308dcf40 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.645501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744af80f-b917-4257-be28-fb1d308dcf40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.649151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744af80f-b917-4257-be28-fb1d308dcf40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:27.650709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744af80f-b917-4257-be28-fb1d308dcf40 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:27.652963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 146/274] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:27.660221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744af80f-b917-4257-be28-fb1d308dcf40 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:27.692046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e0b5e-ef7a-4d99-a643-dcd2e1a75563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:27.692697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e0b5e-ef7a-4d99-a643-dcd2e1a75563 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.692975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e0b5e-ef7a-4d99-a643-dcd2e1a75563 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.693243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e0b5e-ef7a-4d99-a643-dcd2e1a75563 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.715724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e0b5e-ef7a-4d99-a643-dcd2e1a75563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a48e0b5e-ef7a-4d99-a643-dcd2e1a75563 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:27.721991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:27.722895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:27.723667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 129/275] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:27.732394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e0b5e-ef7a-4d99-a643-dcd2e1a75563 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:27.824578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744af80f-b917-4257-be28-fb1d308dcf40 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:27.859551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:27.869991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab726d-6ce2-4d2d-ba51-383b655d6ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:27.870603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab726d-6ce2-4d2d-ba51-383b655d6ef3 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.874457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab726d-6ce2-4d2d-ba51-383b655d6ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.874457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab726d-6ce2-4d2d-ba51-383b655d6ef3 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.874457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab726d-6ce2-4d2d-ba51-383b655d6ef3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:27.874670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e0b5e-ef7a-4d99-a643-dcd2e1a75563 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:27.874741 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab726d-6ce2-4d2d-ba51-383b655d6ef3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:27.902949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:27.908860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1dfada-1587-4516-bf3b-b0a9a4686d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:27.909469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dfada-1587-4516-bf3b-b0a9a4686d1e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.909740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dfada-1587-4516-bf3b-b0a9a4686d1e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.910055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dfada-1587-4516-bf3b-b0a9a4686d1e None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.910718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dfada-1587-4516-bf3b-b0a9a4686d1e None admin] RBAC: Adding query filter params (name=member) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:27.911014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dfada-1587-4516-bf3b-b0a9a4686d1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:27.912396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dfada-1587-4516-bf3b-b0a9a4686d1e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:27.921394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 130/276] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:27.926139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcfc39-f408-4130-8672-49f73dd98860 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.926139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcfc39-f408-4130-8672-49f73dd98860 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.926139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcfc39-f408-4130-8672-49f73dd98860 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:27.926139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 131/277] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:27.926891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcfc39-f408-4130-8672-49f73dd98860 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:27.970582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 147/278] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:27 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:27.979360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e5eb3-c9bc-4601-a826-2c0762568c51 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:27.979640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e5eb3-c9bc-4601-a826-2c0762568c51 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:27.979916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e5eb3-c9bc-4601-a826-2c0762568c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.014304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9f2e5eb3-c9bc-4601-a826-2c0762568c51 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:28.016214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9f2e5eb3-c9bc-4601-a826-2c0762568c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:28.042927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcfc39-f408-4130-8672-49f73dd98860 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:28.089357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58588c4a-d3ed-4a8b-9857-1dfc794a5be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:28.091085 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58588c4a-d3ed-4a8b-9857-1dfc794a5be4 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:28.091577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58588c4a-d3ed-4a8b-9857-1dfc794a5be4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:28.092016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58588c4a-d3ed-4a8b-9857-1dfc794a5be4 None admin] PATH_INFO: `/v3/users/d0f976ca126c446fab4a5c881ceeb922` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.110018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58588c4a-d3ed-4a8b-9857-1dfc794a5be4 None admin] RBAC: Authorizing `identity:get_user(user_id=d0f976ca126c446fab4a5c881ceeb922)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:28.112123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58588c4a-d3ed-4a8b-9857-1dfc794a5be4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:28.113830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 148/279] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:28.119069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 132/280] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:27 2022] GET /identity/v3/users/d0f976ca126c446fab4a5c881ceeb922 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:28.125653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e5eb3-c9bc-4601-a826-2c0762568c51 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:28.135579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58588c4a-d3ed-4a8b-9857-1dfc794a5be4 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:28.188621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:28.255292 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e5eb3-c9bc-4601-a826-2c0762568c51 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:28.265955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58588c4a-d3ed-4a8b-9857-1dfc794a5be4 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:28.273067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:28.297736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caf832-252e-4d39-8acb-de3db7ca4a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:28.298811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf832-252e-4d39-8acb-de3db7ca4a1d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:28.298811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf832-252e-4d39-8acb-de3db7ca4a1d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:28.298985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf832-252e-4d39-8acb-de3db7ca4a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.299791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63caf832-252e-4d39-8acb-de3db7ca4a1d None admin] RBAC: Adding query filter params (role.id=d01bf1c8bfca4691affaddcd9487c751, user.id=d0f976ca126c446fab4a5c881ceeb922, scope.system=all) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:28.300103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63caf832-252e-4d39-8acb-de3db7ca4a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:28.301591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63caf832-252e-4d39-8acb-de3db7ca4a1d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:28.308768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 149/281] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Aug 8 16:40:28 2022] GET /identity/v3/role_assignments?user.id=d0f976ca126c446fab4a5c881ceeb922&scope.system=all&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:28.309890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfac231-be37-4661-bc03-c02dca981503 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:28.310164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfac231-be37-4661-bc03-c02dca981503 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:28.310435 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfac231-be37-4661-bc03-c02dca981503 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.315827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 150/282] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:28.316254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4aab-8016-4a45-8bfe-bb3a758f1b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:28.316918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4aab-8016-4a45-8bfe-bb3a758f1b44 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:28.317217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4aab-8016-4a45-8bfe-bb3a758f1b44 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:28.317448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4aab-8016-4a45-8bfe-bb3a758f1b44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.319244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db4cee-703a-4f92-8074-4ed0c8a461fc None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:28.319591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db4cee-703a-4f92-8074-4ed0c8a461fc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:28.319865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db4cee-703a-4f92-8074-4ed0c8a461fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.331400 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e4aab-8016-4a45-8bfe-bb3a758f1b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-4b8e4aab-8016-4a45-8bfe-bb3a758f1b44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:28.337394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:28.338491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:28.339254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 133/283] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:28.345549 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4aab-8016-4a45-8bfe-bb3a758f1b44 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:28.351685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-75db4cee-703a-4f92-8074-4ed0c8a461fc None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:28.353238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-75db4cee-703a-4f92-8074-4ed0c8a461fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:28.435263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 151/284] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:28.465672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4aab-8016-4a45-8bfe-bb3a758f1b44 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:28.468451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:28.479555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db4cee-703a-4f92-8074-4ed0c8a461fc None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:28.507366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a042937-814d-47bf-9a9b-420ac9bfec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:28.507983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a042937-814d-47bf-9a9b-420ac9bfec1d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:28.508257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a042937-814d-47bf-9a9b-420ac9bfec1d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:28.508528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a042937-814d-47bf-9a9b-420ac9bfec1d None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.509199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a042937-814d-47bf-9a9b-420ac9bfec1d None admin] RBAC: Adding query filter params (name=reader) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:28.509494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a042937-814d-47bf-9a9b-420ac9bfec1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:28.510904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a042937-814d-47bf-9a9b-420ac9bfec1d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:28.521307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 134/285] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:28.532721 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a042937-814d-47bf-9a9b-420ac9bfec1d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:28.624548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db4cee-703a-4f92-8074-4ed0c8a461fc None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:28.635146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:28.664595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96071b51-a64c-4baa-997b-3b392fd9302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:28.665219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96071b51-a64c-4baa-997b-3b392fd9302f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:28.665862 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96071b51-a64c-4baa-997b-3b392fd9302f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:28.666187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96071b51-a64c-4baa-997b-3b392fd9302f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.677878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96071b51-a64c-4baa-997b-3b392fd9302f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-96071b51-a64c-4baa-997b-3b392fd9302f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:28.683030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:28.683925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:28.684712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 152/286] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:28.694920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96071b51-a64c-4baa-997b-3b392fd9302f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:28.751851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a042937-814d-47bf-9a9b-420ac9bfec1d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:28.822493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43729c9b-2658-4a80-812b-970d62c963c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:28.823557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43729c9b-2658-4a80-812b-970d62c963c4 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:28.824430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43729c9b-2658-4a80-812b-970d62c963c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:28.824713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43729c9b-2658-4a80-812b-970d62c963c4 None admin] PATH_INFO: `/v3/users/d11f4676e1a24dae86e0730e6625beaf` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.826530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43729c9b-2658-4a80-812b-970d62c963c4 None admin] RBAC: Authorizing `identity:get_user(user_id=d11f4676e1a24dae86e0730e6625beaf)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:28.828201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43729c9b-2658-4a80-812b-970d62c963c4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:28.833833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 135/287] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:28 2022] GET /identity/v3/users/d11f4676e1a24dae86e0730e6625beaf => generated 297 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:28.839471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43729c9b-2658-4a80-812b-970d62c963c4 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:28.868529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96071b51-a64c-4baa-997b-3b392fd9302f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:28.877145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:28.934046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4a5f7-c299-46ea-b052-01f50528c195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:28.935078 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4a5f7-c299-46ea-b052-01f50528c195 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:28.937568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4a5f7-c299-46ea-b052-01f50528c195 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:28.938006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4a5f7-c299-46ea-b052-01f50528c195 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:28.938986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4a5f7-c299-46ea-b052-01f50528c195 None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:28.940272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4a5f7-c299-46ea-b052-01f50528c195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:28.941961 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4a5f7-c299-46ea-b052-01f50528c195 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:28.958898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 153/288] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:28.965938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4a5f7-c299-46ea-b052-01f50528c195 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:29.007159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43729c9b-2658-4a80-812b-970d62c963c4 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:29.017810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:29.069343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6d217-88ae-4ce2-a119-58bbc78d7f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:29.070038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6d217-88ae-4ce2-a119-58bbc78d7f9d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:29.070323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6d217-88ae-4ce2-a119-58bbc78d7f9d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:29.070595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6d217-88ae-4ce2-a119-58bbc78d7f9d None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:29.073500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6d217-88ae-4ce2-a119-58bbc78d7f9d None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:29.075855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6d217-88ae-4ce2-a119-58bbc78d7f9d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:29.077516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 136/289] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:28 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:29.085508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6d217-88ae-4ce2-a119-58bbc78d7f9d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:29.145980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4a5f7-c299-46ea-b052-01f50528c195 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:29.198018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bc339-5a70-45c9-8f0f-6fc5b38c5a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:29.199044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bc339-5a70-45c9-8f0f-6fc5b38c5a25 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:29.199519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bc339-5a70-45c9-8f0f-6fc5b38c5a25 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:29.200242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bc339-5a70-45c9-8f0f-6fc5b38c5a25 None admin] PATH_INFO: `/v3/users/7ead1cdc2cd446e3ad11b8b89423b907` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:29.202583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bc339-5a70-45c9-8f0f-6fc5b38c5a25 None admin] RBAC: Authorizing `identity:get_user(user_id=7ead1cdc2cd446e3ad11b8b89423b907)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:29.203255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6d217-88ae-4ce2-a119-58bbc78d7f9d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:29.204694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bc339-5a70-45c9-8f0f-6fc5b38c5a25 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:29.206106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:29.207584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 154/290] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:28 2022] GET /identity/v3/users/7ead1cdc2cd446e3ad11b8b89423b907 => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:29.221608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bc339-5a70-45c9-8f0f-6fc5b38c5a25 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:29.267964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9fe11-b281-4a0b-8814-6ca9acb198f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:29.268590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9fe11-b281-4a0b-8814-6ca9acb198f1 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:29.268810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9fe11-b281-4a0b-8814-6ca9acb198f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:29.269081 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9fe11-b281-4a0b-8814-6ca9acb198f1 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd/users/d11f4676e1a24dae86e0730e6625beaf/roles/63d2a6efa7784a7eab9594616b3e8978` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:29.272691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9fe11-b281-4a0b-8814-6ca9acb198f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=801abb9eafc94aef9a541cb855fb81cd, user_id=d11f4676e1a24dae86e0730e6625beaf, role_id=63d2a6efa7784a7eab9594616b3e8978)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:29.274140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9fe11-b281-4a0b-8814-6ca9acb198f1 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:29.330725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 137/291] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:40:29 2022] PUT /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd/users/d11f4676e1a24dae86e0730e6625beaf/roles/63d2a6efa7784a7eab9594616b3e8978 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:40:29.357714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:29.374268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bc339-5a70-45c9-8f0f-6fc5b38c5a25 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:29.421094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a47fad-34e5-4e4a-a3f5-85b52ba5212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:29.421711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47fad-34e5-4e4a-a3f5-85b52ba5212a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:29.422007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47fad-34e5-4e4a-a3f5-85b52ba5212a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:29.422326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47fad-34e5-4e4a-a3f5-85b52ba5212a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:29.508524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47fad-34e5-4e4a-a3f5-85b52ba5212a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:29.510003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47fad-34e5-4e4a-a3f5-85b52ba5212a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:29.616612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:29.621502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 155/292] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 16:40:29.627163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9fe11-b281-4a0b-8814-6ca9acb198f1 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:29.737901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9fe11-b281-4a0b-8814-6ca9acb198f1 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:29.782280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4a2a2-e726-4ee4-a071-5cf80a92c6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:29.782974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4a2a2-e726-4ee4-a071-5cf80a92c6cd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:29.783251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4a2a2-e726-4ee4-a071-5cf80a92c6cd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:29.783521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4a2a2-e726-4ee4-a071-5cf80a92c6cd None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:29.790358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4a2a2-e726-4ee4-a071-5cf80a92c6cd None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:29.791793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4a2a2-e726-4ee4-a071-5cf80a92c6cd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:29.794182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 138/293] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:29 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:29.799934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a47fad-34e5-4e4a-a3f5-85b52ba5212a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:29.917375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a47fad-34e5-4e4a-a3f5-85b52ba5212a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:29.962577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe839ee-1359-497f-894a-5451b2167601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:29.963185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe839ee-1359-497f-894a-5451b2167601 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:29.963458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe839ee-1359-497f-894a-5451b2167601 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:29.963722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe839ee-1359-497f-894a-5451b2167601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:29.965159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe839ee-1359-497f-894a-5451b2167601 None admin] RBAC: Adding query filter params (role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=801abb9eafc94aef9a541cb855fb81cd, user.id=7ead1cdc2cd446e3ad11b8b89423b907) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:29.965459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe839ee-1359-497f-894a-5451b2167601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:29.966906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe839ee-1359-497f-894a-5451b2167601 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:29.975668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 156/294] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:29 2022] GET /identity/v3/role_assignments?user.id=7ead1cdc2cd446e3ad11b8b89423b907&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:30.477288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f28f8-2dce-46bf-bf4a-cf54be509eb8 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:30.477288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f28f8-2dce-46bf-bf4a-cf54be509eb8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:30.477288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f28f8-2dce-46bf-bf4a-cf54be509eb8 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:30.478591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 139/295] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:30.485459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ca52e-38b9-4f19-9129-3cda741ef04b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:30.486115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ca52e-38b9-4f19-9129-3cda741ef04b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:30.486598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ca52e-38b9-4f19-9129-3cda741ef04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:30.510121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dc0ca52e-38b9-4f19-9129-3cda741ef04b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:30.512296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dc0ca52e-38b9-4f19-9129-3cda741ef04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:30.575082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab599acc-8ab0-4827-b4c6-32e2117b0626 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:30.575667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab599acc-8ab0-4827-b4c6-32e2117b0626 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:30.575886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab599acc-8ab0-4827-b4c6-32e2117b0626 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:30.577288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 140/296] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:40:30.582706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536202d1-ba0b-4245-9042-63d93fd980e4 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:30.582983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536202d1-ba0b-4245-9042-63d93fd980e4 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:30.583253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536202d1-ba0b-4245-9042-63d93fd980e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:30.592740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:30.603833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 157/297] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:30.607828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-536202d1-ba0b-4245-9042-63d93fd980e4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:30.609185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-536202d1-ba0b-4245-9042-63d93fd980e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:30.629068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ca52e-38b9-4f19-9129-3cda741ef04b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:30.633899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:30.688158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 141/298] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:30.712251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536202d1-ba0b-4245-9042-63d93fd980e4 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:30.739687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ca52e-38b9-4f19-9129-3cda741ef04b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:30.775763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aa886-9856-49ea-b26c-08607b06be9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:30.776392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aa886-9856-49ea-b26c-08607b06be9a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:30.776669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aa886-9856-49ea-b26c-08607b06be9a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:30.777636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aa886-9856-49ea-b26c-08607b06be9a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:30.788871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661aa886-9856-49ea-b26c-08607b06be9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-661aa886-9856-49ea-b26c-08607b06be9a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:30.793692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:30.794620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:30.795409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 158/299] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:30.800995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aa886-9856-49ea-b26c-08607b06be9a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:30.820073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536202d1-ba0b-4245-9042-63d93fd980e4 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:30.858663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1add7-f41f-4253-ab73-a8292fcd29d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:30.859018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:30.860005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1add7-f41f-4253-ab73-a8292fcd29d9 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:30.860285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1add7-f41f-4253-ab73-a8292fcd29d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:30.860549 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1add7-f41f-4253-ab73-a8292fcd29d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:30.861712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1add7-f41f-4253-ab73-a8292fcd29d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:30.863408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1add7-f41f-4253-ab73-a8292fcd29d9 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:30.865335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 142/300] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:30.866832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934ec5e-d42e-4f36-895a-ea3cf41c5511 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:30.867118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934ec5e-d42e-4f36-895a-ea3cf41c5511 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:30.867389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934ec5e-d42e-4f36-895a-ea3cf41c5511 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:30.873026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 143/301] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:30.873026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934ec5e-d42e-4f36-895a-ea3cf41c5511 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:30.900840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:30.922793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aa886-9856-49ea-b26c-08607b06be9a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:30.960255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1f198-e659-4ba1-ac80-45f8c291a664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:30.961145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1f198-e659-4ba1-ac80-45f8c291a664 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:30.961424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1f198-e659-4ba1-ac80-45f8c291a664 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:30.961695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1f198-e659-4ba1-ac80-45f8c291a664 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:30.962390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1f198-e659-4ba1-ac80-45f8c291a664 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:30.962693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1f198-e659-4ba1-ac80-45f8c291a664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:30.965063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1f198-e659-4ba1-ac80-45f8c291a664 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:30.977551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 159/302] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:30.977551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ac36d-30de-41b5-93b6-3e8860205590 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:30.977551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ac36d-30de-41b5-93b6-3e8860205590 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:30.977551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ac36d-30de-41b5-93b6-3e8860205590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:30.981983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934ec5e-d42e-4f36-895a-ea3cf41c5511 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:30.993164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dd3ac36d-30de-41b5-93b6-3e8860205590 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:30.994298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dd3ac36d-30de-41b5-93b6-3e8860205590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:31.020642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa52093-1434-44b0-be4d-f2bcf94b7310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:31.021311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa52093-1434-44b0-be4d-f2bcf94b7310 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:31.021730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa52093-1434-44b0-be4d-f2bcf94b7310 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:31.022035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa52093-1434-44b0-be4d-f2bcf94b7310 None admin] PATH_INFO: `/v3/groups` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:31.023016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa52093-1434-44b0-be4d-f2bcf94b7310 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:31.024946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa52093-1434-44b0-be4d-f2bcf94b7310 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:31.061075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 160/303] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:31.061928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ac36d-30de-41b5-93b6-3e8860205590 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:31.095583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 144/304] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Aug 8 16:40:30 2022] POST /identity/v3/groups => generated 229 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:31.096763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa52093-1434-44b0-be4d-f2bcf94b7310 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:31.127794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:31.209441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ac36d-30de-41b5-93b6-3e8860205590 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:31.214498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:31.228105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa52093-1434-44b0-be4d-f2bcf94b7310 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:31.259544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5432d-eb5b-4626-9e49-687b007b308f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:31.261762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5432d-eb5b-4626-9e49-687b007b308f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:31.261762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5432d-eb5b-4626-9e49-687b007b308f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:31.261762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5432d-eb5b-4626-9e49-687b007b308f None admin] PATH_INFO: `/v3/users/73ec0a11c5a44010a3f1efee1f76d16f` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:31.279108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5432d-eb5b-4626-9e49-687b007b308f None admin] RBAC: Authorizing `identity:get_user(user_id=73ec0a11c5a44010a3f1efee1f76d16f)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:31.280583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5432d-eb5b-4626-9e49-687b007b308f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:31.281909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac6645-5208-4fff-9ddd-a2e0e16eb8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:31.282606 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 161/305] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:31 2022] GET /identity/v3/users/73ec0a11c5a44010a3f1efee1f76d16f => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:31.283135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac6645-5208-4fff-9ddd-a2e0e16eb8ed None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:31.283684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac6645-5208-4fff-9ddd-a2e0e16eb8ed None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:31.284129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac6645-5208-4fff-9ddd-a2e0e16eb8ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:31.290240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5432d-eb5b-4626-9e49-687b007b308f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:31.301013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac6645-5208-4fff-9ddd-a2e0e16eb8ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-e5ac6645-5208-4fff-9ddd-a2e0e16eb8ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:31.304946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:31.306020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:31.306884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:31.312125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 145/306] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:31.317726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac6645-5208-4fff-9ddd-a2e0e16eb8ed None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:31.441217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac6645-5208-4fff-9ddd-a2e0e16eb8ed None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:31.444526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5432d-eb5b-4626-9e49-687b007b308f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:31.445833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:31.478242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc7cc0-336c-408d-9b17-91e52c042a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:31.479509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc7cc0-336c-408d-9b17-91e52c042a24 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:31.480536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc7cc0-336c-408d-9b17-91e52c042a24 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:31.480950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc7cc0-336c-408d-9b17-91e52c042a24 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:31.481829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc7cc0-336c-408d-9b17-91e52c042a24 None admin] RBAC: Adding query filter params (name=member) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:31.482299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc7cc0-336c-408d-9b17-91e52c042a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:31.483870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc7cc0-336c-408d-9b17-91e52c042a24 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:31.484367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6871e6-bca9-49a4-bdff-9e290dbd3544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:31.485101 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6871e6-bca9-49a4-bdff-9e290dbd3544 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:31.485440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6871e6-bca9-49a4-bdff-9e290dbd3544 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:31.485839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6871e6-bca9-49a4-bdff-9e290dbd3544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:31.486749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6871e6-bca9-49a4-bdff-9e290dbd3544 None admin] RBAC: Adding query filter params (role.id=63d2a6efa7784a7eab9594616b3e8978, user.id=73ec0a11c5a44010a3f1efee1f76d16f, scope.system=all) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:31.487117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6871e6-bca9-49a4-bdff-9e290dbd3544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:31.488720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6871e6-bca9-49a4-bdff-9e290dbd3544 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:31.494368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 162/307] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Aug 8 16:40:31 2022] GET /identity/v3/role_assignments?user.id=73ec0a11c5a44010a3f1efee1f76d16f&scope.system=all&role.id=63d2a6efa7784a7eab9594616b3e8978 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:31.495036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 146/308] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:31.502339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6871e6-bca9-49a4-bdff-9e290dbd3544 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:31.619564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6871e6-bca9-49a4-bdff-9e290dbd3544 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:31.658105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2de37-79d2-46a6-9789-46cb922474c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:31.658791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2de37-79d2-46a6-9789-46cb922474c0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:31.659073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2de37-79d2-46a6-9789-46cb922474c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:31.659345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2de37-79d2-46a6-9789-46cb922474c0 None admin] PATH_INFO: `/v3/users/d0f976ca126c446fab4a5c881ceeb922` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:31.660490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2de37-79d2-46a6-9789-46cb922474c0 None admin] RBAC: Authorizing `identity:get_user(user_id=d0f976ca126c446fab4a5c881ceeb922)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:31.661958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2de37-79d2-46a6-9789-46cb922474c0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:31.668870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 163/309] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:31 2022] GET /identity/v3/users/d0f976ca126c446fab4a5c881ceeb922 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:31.670305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc7cc0-336c-408d-9b17-91e52c042a24 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:31.694174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:31.775355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc7cc0-336c-408d-9b17-91e52c042a24 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:31.797655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791025e7-19ba-493b-b27f-4e067b2f26ec None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:31.798343 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791025e7-19ba-493b-b27f-4e067b2f26ec None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:31.798813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791025e7-19ba-493b-b27f-4e067b2f26ec None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:31.800337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 164/310] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:40:31.805430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e6e99-f7ef-43e8-807c-9819f84b8181 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:31.805979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e6e99-f7ef-43e8-807c-9819f84b8181 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:31.806465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e6e99-f7ef-43e8-807c-9819f84b8181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:31.812930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbde71-2a77-46f6-b33d-eb883719d5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:31.813964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbde71-2a77-46f6-b33d-eb883719d5f3 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:31.814244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbde71-2a77-46f6-b33d-eb883719d5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:31.814515 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbde71-2a77-46f6-b33d-eb883719d5f3 None admin] PATH_INFO: `/v3/system/users/d0f976ca126c446fab4a5c881ceeb922/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:31.816978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbde71-2a77-46f6-b33d-eb883719d5f3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d0f976ca126c446fab4a5c881ceeb922, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:31.818408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbde71-2a77-46f6-b33d-eb883719d5f3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:31.827731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ea3e6e99-f7ef-43e8-807c-9819f84b8181 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:31.828966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ea3e6e99-f7ef-43e8-807c-9819f84b8181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:31.858491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:31.898066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 165/311] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:31.921650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e6e99-f7ef-43e8-807c-9819f84b8181 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:31.924359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 147/312] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Aug 8 16:40:31 2022] PUT /identity/v3/system/users/d0f976ca126c446fab4a5c881ceeb922/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:40:32.038484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e6e99-f7ef-43e8-807c-9819f84b8181 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:32.076316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f57650-08ff-4b80-ab26-baeb476ab271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:32.076955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f57650-08ff-4b80-ab26-baeb476ab271 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:32.077231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f57650-08ff-4b80-ab26-baeb476ab271 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:32.077550 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f57650-08ff-4b80-ab26-baeb476ab271 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f57650-08ff-4b80-ab26-baeb476ab271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-b5f57650-08ff-4b80-ab26-baeb476ab271 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:32.097817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:32.098700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:32.099530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:32.101656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 166/313] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:32.108821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbde71-2a77-46f6-b33d-eb883719d5f3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:32.192946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:32.219038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbde71-2a77-46f6-b33d-eb883719d5f3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:32.255493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2498e38-8149-4819-b55e-f27589435617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:32.256109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2498e38-8149-4819-b55e-f27589435617 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:32.256392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2498e38-8149-4819-b55e-f27589435617 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:32.256664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2498e38-8149-4819-b55e-f27589435617 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:32.257337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2498e38-8149-4819-b55e-f27589435617 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:32.257640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2498e38-8149-4819-b55e-f27589435617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:32.259062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2498e38-8149-4819-b55e-f27589435617 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:32.268470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 148/314] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:32.275058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f57650-08ff-4b80-ab26-baeb476ab271 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:32.306470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:32.388432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f57650-08ff-4b80-ab26-baeb476ab271 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:32.423349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44138e0b-17be-46b1-a522-32131b95b5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:32.423976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44138e0b-17be-46b1-a522-32131b95b5bd None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:32.424255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44138e0b-17be-46b1-a522-32131b95b5bd None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:32.424527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44138e0b-17be-46b1-a522-32131b95b5bd None admin] PATH_INFO: `/v3/users/d11f4676e1a24dae86e0730e6625beaf` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:32.425656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44138e0b-17be-46b1-a522-32131b95b5bd None admin] RBAC: Authorizing `identity:get_user(user_id=d11f4676e1a24dae86e0730e6625beaf)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:32.427127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44138e0b-17be-46b1-a522-32131b95b5bd None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:32.428882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 167/315] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:32 2022] GET /identity/v3/users/d11f4676e1a24dae86e0730e6625beaf => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:32.435033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2498e38-8149-4819-b55e-f27589435617 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:32.540388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2498e38-8149-4819-b55e-f27589435617 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:32.578972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3437b0-aacb-4b86-8715-a73eebe84192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:32.579946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3437b0-aacb-4b86-8715-a73eebe84192 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:32.580442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3437b0-aacb-4b86-8715-a73eebe84192 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:32.580907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3437b0-aacb-4b86-8715-a73eebe84192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:32.585423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:32.666291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3437b0-aacb-4b86-8715-a73eebe84192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:32.668345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3437b0-aacb-4b86-8715-a73eebe84192 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:32.771961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 149/316] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:32.777956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44138e0b-17be-46b1-a522-32131b95b5bd None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:32.873205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:32.882077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44138e0b-17be-46b1-a522-32131b95b5bd None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:32.916970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b10d3-c2ba-4c65-8ed0-d253cdf053f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:32.918069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b10d3-c2ba-4c65-8ed0-d253cdf053f8 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:32.918350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b10d3-c2ba-4c65-8ed0-d253cdf053f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:32.918618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b10d3-c2ba-4c65-8ed0-d253cdf053f8 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:32.919744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b10d3-c2ba-4c65-8ed0-d253cdf053f8 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:32.921190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b10d3-c2ba-4c65-8ed0-d253cdf053f8 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:32.927876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 168/317] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:32 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:32.933930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3437b0-aacb-4b86-8715-a73eebe84192 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:32.969236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:33.039044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3437b0-aacb-4b86-8715-a73eebe84192 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:33.074843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ba0d1-4292-458d-a17f-90496a4951dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:33.075702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ba0d1-4292-458d-a17f-90496a4951dd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:33.075979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ba0d1-4292-458d-a17f-90496a4951dd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:33.076259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ba0d1-4292-458d-a17f-90496a4951dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:33.076978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ba0d1-4292-458d-a17f-90496a4951dd None admin] RBAC: Adding query filter params (role.id=63d2a6efa7784a7eab9594616b3e8978, scope.project.id=801abb9eafc94aef9a541cb855fb81cd, user.id=d11f4676e1a24dae86e0730e6625beaf) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:33.077281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ba0d1-4292-458d-a17f-90496a4951dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:33.078730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ba0d1-4292-458d-a17f-90496a4951dd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:33.084720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 150/318] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:32 2022] GET /identity/v3/role_assignments?user.id=d11f4676e1a24dae86e0730e6625beaf&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=63d2a6efa7784a7eab9594616b3e8978 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:33.208723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd092e-427b-4184-b84a-f577fe7b8404 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:33.208909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd092e-427b-4184-b84a-f577fe7b8404 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:33.209180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd092e-427b-4184-b84a-f577fe7b8404 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:33.210875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 169/319] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:33.217048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c42f73-bd54-4520-a274-e0eab76ad335 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:33.217330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c42f73-bd54-4520-a274-e0eab76ad335 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:33.217605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c42f73-bd54-4520-a274-e0eab76ad335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:33.236826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-10c42f73-bd54-4520-a274-e0eab76ad335 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:33.237932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-10c42f73-bd54-4520-a274-e0eab76ad335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:33.315019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 151/320] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:33.338400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd092e-427b-4184-b84a-f577fe7b8404 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:33.450032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd092e-427b-4184-b84a-f577fe7b8404 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:33.484488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2695bbc-fecb-4406-9f22-a9edb0faff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:33.485119 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695bbc-fecb-4406-9f22-a9edb0faff25 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:33.485397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695bbc-fecb-4406-9f22-a9edb0faff25 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:33.485667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695bbc-fecb-4406-9f22-a9edb0faff25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:33.486848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2695bbc-fecb-4406-9f22-a9edb0faff25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:33.488281 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2695bbc-fecb-4406-9f22-a9edb0faff25 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:33.494629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 170/321] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:33.495902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c42f73-bd54-4520-a274-e0eab76ad335 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:33.517030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295790c-0eb5-421a-abe3-aa21f82be7bb None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:33.517331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295790c-0eb5-421a-abe3-aa21f82be7bb None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:33.517580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295790c-0eb5-421a-abe3-aa21f82be7bb None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:33.519700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 171/322] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:40:33.525318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef4da7-c71e-4998-9492-a6e8773dc2e9 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:33.525593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef4da7-c71e-4998-9492-a6e8773dc2e9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:33.525889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef4da7-c71e-4998-9492-a6e8773dc2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:33.528028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:33.546125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-80ef4da7-c71e-4998-9492-a6e8773dc2e9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:33.547293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-80ef4da7-c71e-4998-9492-a6e8773dc2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:33.570703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:33.603172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c42f73-bd54-4520-a274-e0eab76ad335 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:33.629960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 172/323] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:33.638725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23e46d-1793-46aa-a906-24297a3fefc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:33.639383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23e46d-1793-46aa-a906-24297a3fefc8 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:33.639659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23e46d-1793-46aa-a906-24297a3fefc8 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:33.639923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23e46d-1793-46aa-a906-24297a3fefc8 None admin] PATH_INFO: `/v3/groups` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:33.640534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23e46d-1793-46aa-a906-24297a3fefc8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:33.641989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23e46d-1793-46aa-a906-24297a3fefc8 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:33.650436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef4da7-c71e-4998-9492-a6e8773dc2e9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:33.719319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 152/324] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Aug 8 16:40:33 2022] POST /identity/v3/groups => generated 226 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:33.766719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef4da7-c71e-4998-9492-a6e8773dc2e9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:33.803438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024433b-f640-4a26-957f-e882bf445ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:33.804094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024433b-f640-4a26-957f-e882bf445ec1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:33.804325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024433b-f640-4a26-957f-e882bf445ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:33.804607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024433b-f640-4a26-957f-e882bf445ec1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:33.815930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024433b-f640-4a26-957f-e882bf445ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-a024433b-f640-4a26-957f-e882bf445ec1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:33.820685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:33.821560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:33.822389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 173/325] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:33.829579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23e46d-1793-46aa-a906-24297a3fefc8 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:33.928229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:33.938039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23e46d-1793-46aa-a906-24297a3fefc8 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:33.975093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd55b4e-835b-4b00-a0e3-ce84405aa03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:33.975806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd55b4e-835b-4b00-a0e3-ce84405aa03c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:33.976150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd55b4e-835b-4b00-a0e3-ce84405aa03c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:33.976497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd55b4e-835b-4b00-a0e3-ce84405aa03c None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:33.978765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd55b4e-835b-4b00-a0e3-ce84405aa03c None admin] RBAC: Adding query filter params (name=reader) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:33.978765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd55b4e-835b-4b00-a0e3-ce84405aa03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:33.978947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd55b4e-835b-4b00-a0e3-ce84405aa03c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:33.989110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 153/326] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:33.994739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024433b-f640-4a26-957f-e882bf445ec1 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:34.018168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:34.105518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024433b-f640-4a26-957f-e882bf445ec1 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:34.118493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3e5a9-3558-432d-8864-2e729123348a None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:34.118780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3e5a9-3558-432d-8864-2e729123348a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:34.119044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3e5a9-3558-432d-8864-2e729123348a None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:34.124968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 154/327] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:34.129487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd690807-faa2-4814-a8c4-0d083a861d21 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:34.129487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd690807-faa2-4814-a8c4-0d083a861d21 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:34.129487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd690807-faa2-4814-a8c4-0d083a861d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:34.146923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82893907-f2af-453b-8ae2-9a542576e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:34.147556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82893907-f2af-453b-8ae2-9a542576e46f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:34.147837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82893907-f2af-453b-8ae2-9a542576e46f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:34.148103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82893907-f2af-453b-8ae2-9a542576e46f None admin] PATH_INFO: `/v3/users/73ec0a11c5a44010a3f1efee1f76d16f` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:34.149217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82893907-f2af-453b-8ae2-9a542576e46f None admin] RBAC: Authorizing `identity:get_user(user_id=73ec0a11c5a44010a3f1efee1f76d16f)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:34.149955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-bd690807-faa2-4814-a8c4-0d083a861d21 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:34.150705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82893907-f2af-453b-8ae2-9a542576e46f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:34.151153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-bd690807-faa2-4814-a8c4-0d083a861d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:34.152684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 174/328] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:33 2022] GET /identity/v3/users/73ec0a11c5a44010a3f1efee1f76d16f => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:34.158127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82893907-f2af-453b-8ae2-9a542576e46f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:34.239192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 155/329] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:34.262743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:34.263012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd690807-faa2-4814-a8c4-0d083a861d21 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:34.275373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82893907-f2af-453b-8ae2-9a542576e46f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:34.324738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878bb58-6d78-4794-92cd-1dc625ac8b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:34.325655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878bb58-6d78-4794-92cd-1dc625ac8b6e None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:34.325984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878bb58-6d78-4794-92cd-1dc625ac8b6e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:34.326259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878bb58-6d78-4794-92cd-1dc625ac8b6e None admin] PATH_INFO: `/v3/system/users/73ec0a11c5a44010a3f1efee1f76d16f/roles/63d2a6efa7784a7eab9594616b3e8978` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:34.327700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878bb58-6d78-4794-92cd-1dc625ac8b6e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=73ec0a11c5a44010a3f1efee1f76d16f, role_id=63d2a6efa7784a7eab9594616b3e8978)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:34.329089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878bb58-6d78-4794-92cd-1dc625ac8b6e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:34.374191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:34.429070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd690807-faa2-4814-a8c4-0d083a861d21 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:34.467526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 175/330] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Aug 8 16:40:34 2022] PUT /identity/v3/system/users/73ec0a11c5a44010a3f1efee1f76d16f/roles/63d2a6efa7784a7eab9594616b3e8978 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:40:34.468683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9cd4-0b33-49eb-8c16-402300fd2729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:34.469440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9cd4-0b33-49eb-8c16-402300fd2729 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:34.469726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9cd4-0b33-49eb-8c16-402300fd2729 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:34.470022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9cd4-0b33-49eb-8c16-402300fd2729 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:34.484740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d9cd4-0b33-49eb-8c16-402300fd2729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-c90d9cd4-0b33-49eb-8c16-402300fd2729 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:34.489931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:34.490846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:34.491611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:34.494963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 156/331] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:34.497963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878bb58-6d78-4794-92cd-1dc625ac8b6e None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:34.629750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878bb58-6d78-4794-92cd-1dc625ac8b6e None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:34.675185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a35c8a-8f4a-4b4d-a0eb-7b32351c4e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:34.675795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a35c8a-8f4a-4b4d-a0eb-7b32351c4e2a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:34.676071 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a35c8a-8f4a-4b4d-a0eb-7b32351c4e2a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:34.676335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a35c8a-8f4a-4b4d-a0eb-7b32351c4e2a None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:34.677003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a35c8a-8f4a-4b4d-a0eb-7b32351c4e2a None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:34.677304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a35c8a-8f4a-4b4d-a0eb-7b32351c4e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:34.678709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a35c8a-8f4a-4b4d-a0eb-7b32351c4e2a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:34.689950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 176/332] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:34.695388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9cd4-0b33-49eb-8c16-402300fd2729 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:34.817821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9cd4-0b33-49eb-8c16-402300fd2729 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:34.856795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf904-867b-4769-afc6-c95eab6e2575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:34.857354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf904-867b-4769-afc6-c95eab6e2575 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:34.857632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf904-867b-4769-afc6-c95eab6e2575 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:34.857931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf904-867b-4769-afc6-c95eab6e2575 None admin] PATH_INFO: `/v3/users/d0f976ca126c446fab4a5c881ceeb922` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:34.859146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cf904-867b-4769-afc6-c95eab6e2575 None admin] RBAC: Authorizing `identity:get_user(user_id=d0f976ca126c446fab4a5c881ceeb922)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:34.860687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cf904-867b-4769-afc6-c95eab6e2575 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:34.862309 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 157/333] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:34 2022] GET /identity/v3/users/d0f976ca126c446fab4a5c881ceeb922 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:34.868263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a35c8a-8f4a-4b4d-a0eb-7b32351c4e2a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:34.888607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:34.986305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a35c8a-8f4a-4b4d-a0eb-7b32351c4e2a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:35.030054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a61f2-a7a4-4631-9dc1-eda60fa017ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:35.030663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a61f2-a7a4-4631-9dc1-eda60fa017ee None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:35.030940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a61f2-a7a4-4631-9dc1-eda60fa017ee None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:35.031204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a61f2-a7a4-4631-9dc1-eda60fa017ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:35.031939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a61f2-a7a4-4631-9dc1-eda60fa017ee None admin] RBAC: Adding query filter params (role.id=d01bf1c8bfca4691affaddcd9487c751, user.id=d0f976ca126c446fab4a5c881ceeb922, scope.system=all) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:35.032243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a61f2-a7a4-4631-9dc1-eda60fa017ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:35.033669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a61f2-a7a4-4631-9dc1-eda60fa017ee None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:35.049398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 177/334] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Aug 8 16:40:34 2022] GET /identity/v3/role_assignments?user.id=d0f976ca126c446fab4a5c881ceeb922&scope.system=all&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:36.793495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995b495-9191-4480-b557-f74251eda458 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:36.794015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995b495-9191-4480-b557-f74251eda458 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:36.794087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995b495-9191-4480-b557-f74251eda458 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:36.795431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 158/335] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:36.806899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e621b8-e309-4d94-a85d-7eb72fc49304 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:36.807378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e621b8-e309-4d94-a85d-7eb72fc49304 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:36.807452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e621b8-e309-4d94-a85d-7eb72fc49304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:36.831954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-18e621b8-e309-4d94-a85d-7eb72fc49304 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:36.833164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-18e621b8-e309-4d94-a85d-7eb72fc49304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:36.906845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:36.916322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 178/336] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:36.959046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995b495-9191-4480-b557-f74251eda458 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:36.977319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:36.991660 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafe3ff-e665-4a20-8b12-c206e0266436 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:36.991952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafe3ff-e665-4a20-8b12-c206e0266436 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:36.992220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafe3ff-e665-4a20-8b12-c206e0266436 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:36.993563 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 179/337] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:37.002820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b455ed-c1e7-4a8a-8010-73a1cc80a3c4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.003106 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b455ed-c1e7-4a8a-8010-73a1cc80a3c4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.003397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b455ed-c1e7-4a8a-8010-73a1cc80a3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.029687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b3b455ed-c1e7-4a8a-8010-73a1cc80a3c4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:37.030907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b3b455ed-c1e7-4a8a-8010-73a1cc80a3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:37.087217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995b495-9191-4480-b557-f74251eda458 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:37.143345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 180/338] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:37.144607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fe290-b089-44d3-9e34-9c8709e79b93 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.144884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fe290-b089-44d3-9e34-9c8709e79b93 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.145145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fe290-b089-44d3-9e34-9c8709e79b93 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.154647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 181/339] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:37.160598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312682d-0c8c-4ab8-b2d5-ae5de6e0709c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.160813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312682d-0c8c-4ab8-b2d5-ae5de6e0709c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.161039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee14c4-1a0f-4580-b334-bdede2bd10e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:37.161165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312682d-0c8c-4ab8-b2d5-ae5de6e0709c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.162018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee14c4-1a0f-4580-b334-bdede2bd10e9 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.162487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee14c4-1a0f-4580-b334-bdede2bd10e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.162941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee14c4-1a0f-4580-b334-bdede2bd10e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.186603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-7312682d-0c8c-4ab8-b2d5-ae5de6e0709c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:37.187890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7312682d-0c8c-4ab8-b2d5-ae5de6e0709c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:37.194160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee14c4-1a0f-4580-b334-bdede2bd10e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-20ee14c4-1a0f-4580-b334-bdede2bd10e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:37.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:37.201054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:37.201872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:37.210098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 159/340] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:37.211446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee14c4-1a0f-4580-b334-bdede2bd10e9 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:37.264362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 182/341] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:37.265283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312682d-0c8c-4ab8-b2d5-ae5de6e0709c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:37.332423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:37.334665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:37.378334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312682d-0c8c-4ab8-b2d5-ae5de6e0709c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:37.390030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee14c4-1a0f-4580-b334-bdede2bd10e9 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:37.414092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f72e1-6711-42cd-9dcc-1e5dce975355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:37.414625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f72e1-6711-42cd-9dcc-1e5dce975355 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.414898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f72e1-6711-42cd-9dcc-1e5dce975355 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.415163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f72e1-6711-42cd-9dcc-1e5dce975355 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.415883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f72e1-6711-42cd-9dcc-1e5dce975355 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:37.416201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f72e1-6711-42cd-9dcc-1e5dce975355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:37.417698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f72e1-6711-42cd-9dcc-1e5dce975355 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:37.426698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 183/342] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:37.427671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f72e1-6711-42cd-9dcc-1e5dce975355 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:37.459605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ffcd3-9ef6-46bf-ad95-118f3d7d785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:37.460915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ffcd3-9ef6-46bf-ad95-118f3d7d785b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.461469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ffcd3-9ef6-46bf-ad95-118f3d7d785b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.462057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ffcd3-9ef6-46bf-ad95-118f3d7d785b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.486814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ffcd3-9ef6-46bf-ad95-118f3d7d785b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b51ffcd3-9ef6-46bf-ad95-118f3d7d785b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:37.492653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:37.493495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:37.494301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:37.503327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 160/343] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:37.504976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ffcd3-9ef6-46bf-ad95-118f3d7d785b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:37.536038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f72e1-6711-42cd-9dcc-1e5dce975355 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:37.570151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:37.594406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c495a4-1782-464b-9fd1-2835358f6ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:37.595155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c495a4-1782-464b-9fd1-2835358f6ad3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.595429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c495a4-1782-464b-9fd1-2835358f6ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.595699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c495a4-1782-464b-9fd1-2835358f6ad3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.607600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c495a4-1782-464b-9fd1-2835358f6ad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-52c495a4-1782-464b-9fd1-2835358f6ad3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:37.613294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:37.618844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 184/344] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:37.619648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c495a4-1782-464b-9fd1-2835358f6ad3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:37.650224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ffcd3-9ef6-46bf-ad95-118f3d7d785b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:37.716266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:37.725536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6cb3f-8550-4b38-85b8-43e4f974d572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:37.732921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6cb3f-8550-4b38-85b8-43e4f974d572 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.732921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6cb3f-8550-4b38-85b8-43e4f974d572 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.732921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6cb3f-8550-4b38-85b8-43e4f974d572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.805562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c495a4-1782-464b-9fd1-2835358f6ad3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:37.806837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6cb3f-8550-4b38-85b8-43e4f974d572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:37.809062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6cb3f-8550-4b38-85b8-43e4f974d572 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:37.841956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef48e5-0803-482f-a0f3-4df0c6752743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:37.842607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef48e5-0803-482f-a0f3-4df0c6752743 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.842830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef48e5-0803-482f-a0f3-4df0c6752743 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.843099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef48e5-0803-482f-a0f3-4df0c6752743 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.843760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef48e5-0803-482f-a0f3-4df0c6752743 None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:37.844059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef48e5-0803-482f-a0f3-4df0c6752743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:37.845431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef48e5-0803-482f-a0f3-4df0c6752743 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:37.856183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 185/345] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:37.857424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e67da-5943-43d7-9c21-cb48517ff0bb None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.857708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e67da-5943-43d7-9c21-cb48517ff0bb None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.862092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e67da-5943-43d7-9c21-cb48517ff0bb None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.863633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 186/346] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:37.864364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e67da-5943-43d7-9c21-cb48517ff0bb None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:37.918566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:37.923128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 161/347] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:37.924555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca452e5b-754f-4f98-a70d-8dda58af9053 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:37.924991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca452e5b-754f-4f98-a70d-8dda58af9053 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:37.925407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca452e5b-754f-4f98-a70d-8dda58af9053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:37.944541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ca452e5b-754f-4f98-a70d-8dda58af9053 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:37.947316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ca452e5b-754f-4f98-a70d-8dda58af9053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:37.975645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:37.978223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e67da-5943-43d7-9c21-cb48517ff0bb None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:38.015143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb428-7ad2-4d63-8e39-fc8610a2d95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:38.015807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb428-7ad2-4d63-8e39-fc8610a2d95a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.016202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb428-7ad2-4d63-8e39-fc8610a2d95a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.016361 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eb428-7ad2-4d63-8e39-fc8610a2d95a None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.017026 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eb428-7ad2-4d63-8e39-fc8610a2d95a None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:38.017326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eb428-7ad2-4d63-8e39-fc8610a2d95a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:38.019050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eb428-7ad2-4d63-8e39-fc8610a2d95a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:38.027543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 162/348] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:38.028605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca452e5b-754f-4f98-a70d-8dda58af9053 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:38.029022 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 187/349] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:38.029853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb428-7ad2-4d63-8e39-fc8610a2d95a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:38.144767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca452e5b-754f-4f98-a70d-8dda58af9053 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:38.162751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eb428-7ad2-4d63-8e39-fc8610a2d95a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:38.171349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:38.186620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e47ea-01b9-4a5d-bf29-ca4c717cffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:38.187543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e47ea-01b9-4a5d-bf29-ca4c717cffcb None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.187985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e47ea-01b9-4a5d-bf29-ca4c717cffcb None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.188423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e47ea-01b9-4a5d-bf29-ca4c717cffcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.205679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff7a81-e6ed-4992-952d-a3af1b3d6d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:38.206366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff7a81-e6ed-4992-952d-a3af1b3d6d93 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.206647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff7a81-e6ed-4992-952d-a3af1b3d6d93 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.206910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff7a81-e6ed-4992-952d-a3af1b3d6d93 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.208681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff7a81-e6ed-4992-952d-a3af1b3d6d93 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:38.210720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff7a81-e6ed-4992-952d-a3af1b3d6d93 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:38.212865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 188/350] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:38 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:38.216838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907bf8d-6641-4676-9700-36ee93db534f None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.216838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907bf8d-6641-4676-9700-36ee93db534f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.216838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907bf8d-6641-4676-9700-36ee93db534f None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.216838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 189/351] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:38.216838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907bf8d-6641-4676-9700-36ee93db534f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:38.223931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:38.272316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e47ea-01b9-4a5d-bf29-ca4c717cffcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:38.274402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e47ea-01b9-4a5d-bf29-ca4c717cffcb None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:38.335162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907bf8d-6641-4676-9700-36ee93db534f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:38.383942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc14b6-efe7-4ec4-a542-865a6d3ab86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:38.384647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc14b6-efe7-4ec4-a542-865a6d3ab86d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.384820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc14b6-efe7-4ec4-a542-865a6d3ab86d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.385086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc14b6-efe7-4ec4-a542-865a6d3ab86d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.395361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 163/352] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:38.395548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e47ea-01b9-4a5d-bf29-ca4c717cffcb None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:38.434918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:38.461826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc14b6-efe7-4ec4-a542-865a6d3ab86d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:38.461826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc14b6-efe7-4ec4-a542-865a6d3ab86d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:38.470962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:38.520542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e47ea-01b9-4a5d-bf29-ca4c717cffcb None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:38.564634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500b33f-07ed-467d-8e98-6d9add7ea822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:38.565195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500b33f-07ed-467d-8e98-6d9add7ea822 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.565474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500b33f-07ed-467d-8e98-6d9add7ea822 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.566334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500b33f-07ed-467d-8e98-6d9add7ea822 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.579251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500b33f-07ed-467d-8e98-6d9add7ea822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:38.584073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 190/353] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:38.585190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc14b6-efe7-4ec4-a542-865a6d3ab86d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-0500b33f-07ed-467d-8e98-6d9add7ea822 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:38.588501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:38.590230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 164/354] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:38.591065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa388a98-9344-49da-b8b4-128bcdf98dc9 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.591874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa388a98-9344-49da-b8b4-128bcdf98dc9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.591874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa388a98-9344-49da-b8b4-128bcdf98dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.613146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-aa388a98-9344-49da-b8b4-128bcdf98dc9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:38.614373 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-aa388a98-9344-49da-b8b4-128bcdf98dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:38.682906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:38.695828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 165/355] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:38.697120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a4167-ff2a-43a2-97de-f2d1fa87d940 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.698957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a4167-ff2a-43a2-97de-f2d1fa87d940 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.698957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a4167-ff2a-43a2-97de-f2d1fa87d940 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.700245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 166/356] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:38.704285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a4167-ff2a-43a2-97de-f2d1fa87d940 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:38.717530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc14b6-efe7-4ec4-a542-865a6d3ab86d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:38.760021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:38.763260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe605f-76d2-41b1-ae93-d90b0860cc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:38.763905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe605f-76d2-41b1-ae93-d90b0860cc84 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.764186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe605f-76d2-41b1-ae93-d90b0860cc84 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.764459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe605f-76d2-41b1-ae93-d90b0860cc84 None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.771112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe605f-76d2-41b1-ae93-d90b0860cc84 None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:38.772583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe605f-76d2-41b1-ae93-d90b0860cc84 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:38.775065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 191/357] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:38 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:38.775065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe605f-76d2-41b1-ae93-d90b0860cc84 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:38.825925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a4167-ff2a-43a2-97de-f2d1fa87d940 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:38.870236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696df4f8-c9fb-460a-be90-4d9ec5ecd7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:38.870873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696df4f8-c9fb-460a-be90-4d9ec5ecd7ec None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.871330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696df4f8-c9fb-460a-be90-4d9ec5ecd7ec None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.871434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696df4f8-c9fb-460a-be90-4d9ec5ecd7ec None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.872536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696df4f8-c9fb-460a-be90-4d9ec5ecd7ec None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:38.874025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696df4f8-c9fb-460a-be90-4d9ec5ecd7ec None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:38.875652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 167/358] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:38 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:38.876406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696df4f8-c9fb-460a-be90-4d9ec5ecd7ec None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:38.902338 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe605f-76d2-41b1-ae93-d90b0860cc84 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:38.945701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7e759-5fee-45be-976c-e4e192d35ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:38.946375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e759-5fee-45be-976c-e4e192d35ef1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.946656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e759-5fee-45be-976c-e4e192d35ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.946923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e759-5fee-45be-976c-e4e192d35ef1 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.948635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7e759-5fee-45be-976c-e4e192d35ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:38.950633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7e759-5fee-45be-976c-e4e192d35ef1 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:38.953315 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 192/359] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:38 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:38.953587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796dd72-3f42-4449-baee-ced50fb71935 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:38.953773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796dd72-3f42-4449-baee-ced50fb71935 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:38.954068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796dd72-3f42-4449-baee-ced50fb71935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:38.968882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:38.974452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d796dd72-3f42-4449-baee-ced50fb71935 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:38.975664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d796dd72-3f42-4449-baee-ced50fb71935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:38.998328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696df4f8-c9fb-460a-be90-4d9ec5ecd7ec None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.018124 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:39.036191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3898556-1e07-4899-887f-b41a001aae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.036827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3898556-1e07-4899-887f-b41a001aae20 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.037105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3898556-1e07-4899-887f-b41a001aae20 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.037828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3898556-1e07-4899-887f-b41a001aae20 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.038112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3898556-1e07-4899-887f-b41a001aae20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:39.038415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3898556-1e07-4899-887f-b41a001aae20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:39.039783 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3898556-1e07-4899-887f-b41a001aae20 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:39.050330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 168/360] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:39.051064 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3898556-1e07-4899-887f-b41a001aae20 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.069854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 193/361] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:39.069854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796dd72-3f42-4449-baee-ced50fb71935 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.176134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3898556-1e07-4899-887f-b41a001aae20 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.186223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796dd72-3f42-4449-baee-ced50fb71935 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.213372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca450f4a-c8f9-4a28-b3b9-5aba58da2320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.214719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450f4a-c8f9-4a28-b3b9-5aba58da2320 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.214993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450f4a-c8f9-4a28-b3b9-5aba58da2320 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.215256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450f4a-c8f9-4a28-b3b9-5aba58da2320 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.225322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e6feb-9935-4924-9e14-10030c1fa48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.225703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca450f4a-c8f9-4a28-b3b9-5aba58da2320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:39.226003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e6feb-9935-4924-9e14-10030c1fa48f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.226285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e6feb-9935-4924-9e14-10030c1fa48f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.227249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e6feb-9935-4924-9e14-10030c1fa48f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.227249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e6feb-9935-4924-9e14-10030c1fa48f None admin] RBAC: Adding query filter params (group.id=1198d79329074d26a5991f4153c0913c, role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:39.227575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e6feb-9935-4924-9e14-10030c1fa48f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:39.229008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e6feb-9935-4924-9e14-10030c1fa48f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-ca450f4a-c8f9-4a28-b3b9-5aba58da2320 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:39.230674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:39.231534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:39.232311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 169/362] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:39.232964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca450f4a-c8f9-4a28-b3b9-5aba58da2320 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.235250 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:39.237543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 194/363] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/role_assignments?group.id=1198d79329074d26a5991f4153c0913c&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:39.238668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e6feb-9935-4924-9e14-10030c1fa48f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.295232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:39.377488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca450f4a-c8f9-4a28-b3b9-5aba58da2320 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.392001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e6feb-9935-4924-9e14-10030c1fa48f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.430735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb166a0d-fee9-427d-9a7f-1d1ec4139760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.431403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb166a0d-fee9-427d-9a7f-1d1ec4139760 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.431690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb166a0d-fee9-427d-9a7f-1d1ec4139760 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.431965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb166a0d-fee9-427d-9a7f-1d1ec4139760 None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.433692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb166a0d-fee9-427d-9a7f-1d1ec4139760 None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:39.434600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e69e70-c871-47e9-85d4-da24bbfb0800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.435187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb166a0d-fee9-427d-9a7f-1d1ec4139760 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:39.435280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e69e70-c871-47e9-85d4-da24bbfb0800 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.435497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e69e70-c871-47e9-85d4-da24bbfb0800 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.435769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e69e70-c871-47e9-85d4-da24bbfb0800 None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.438399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e69e70-c871-47e9-85d4-da24bbfb0800 None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:39.438506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 170/364] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:39.438506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb166a0d-fee9-427d-9a7f-1d1ec4139760 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.438645 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e69e70-c871-47e9-85d4-da24bbfb0800 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:39.439930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 195/365] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:39.440656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e69e70-c871-47e9-85d4-da24bbfb0800 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.531083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:39.560793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb166a0d-fee9-427d-9a7f-1d1ec4139760 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.568815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e69e70-c871-47e9-85d4-da24bbfb0800 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.570742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:39.605342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4e721-4724-4d4f-967c-286f0662ebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.606046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4e721-4724-4d4f-967c-286f0662ebf7 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.606329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4e721-4724-4d4f-967c-286f0662ebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.606592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4e721-4724-4d4f-967c-286f0662ebf7 None admin] PATH_INFO: `/v3/users/73ec0a11c5a44010a3f1efee1f76d16f` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.607808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4e721-4724-4d4f-967c-286f0662ebf7 None admin] RBAC: Authorizing `identity:get_user(user_id=73ec0a11c5a44010a3f1efee1f76d16f)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:39.609271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4e721-4724-4d4f-967c-286f0662ebf7 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:39.611022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 171/366] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/users/73ec0a11c5a44010a3f1efee1f76d16f => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:39.611815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4e721-4724-4d4f-967c-286f0662ebf7 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.618173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc695d4-63d3-4a9d-b651-7346f06a3fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.619802 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc695d4-63d3-4a9d-b651-7346f06a3fa5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.620740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc695d4-63d3-4a9d-b651-7346f06a3fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.621146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc695d4-63d3-4a9d-b651-7346f06a3fa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.634602 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc695d4-63d3-4a9d-b651-7346f06a3fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-4cc695d4-63d3-4a9d-b651-7346f06a3fa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:39.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:39.642398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:39.643337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:39.645227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 196/367] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:39.646033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc695d4-63d3-4a9d-b651-7346f06a3fa5 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.726184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4e721-4724-4d4f-967c-286f0662ebf7 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.754560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc695d4-63d3-4a9d-b651-7346f06a3fa5 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.762066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a242b-8d40-48fe-9e7b-325fa86560ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.762683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a242b-8d40-48fe-9e7b-325fa86560ae None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.762958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a242b-8d40-48fe-9e7b-325fa86560ae None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.763228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a242b-8d40-48fe-9e7b-325fa86560ae None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.763897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a242b-8d40-48fe-9e7b-325fa86560ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:39.764203 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a242b-8d40-48fe-9e7b-325fa86560ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:39.765564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a242b-8d40-48fe-9e7b-325fa86560ae None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:39.773768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 172/368] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:39.774427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a242b-8d40-48fe-9e7b-325fa86560ae None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.785738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:39.795205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b77159-8a26-472a-b996-c4a78792cc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.796062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b77159-8a26-472a-b996-c4a78792cc1c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.796531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b77159-8a26-472a-b996-c4a78792cc1c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.797506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b77159-8a26-472a-b996-c4a78792cc1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.798782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b77159-8a26-472a-b996-c4a78792cc1c None admin] RBAC: Adding query filter params (group.id=1198d79329074d26a5991f4153c0913c, role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:39.799303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b77159-8a26-472a-b996-c4a78792cc1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:39.802094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b77159-8a26-472a-b996-c4a78792cc1c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:39.810825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 197/369] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/role_assignments?group.id=1198d79329074d26a5991f4153c0913c&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:39.812652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b77159-8a26-472a-b996-c4a78792cc1c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.816792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:39.896583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a242b-8d40-48fe-9e7b-325fa86560ae None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:39.935261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b716f8c-e393-4e45-b007-08f1816d0e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:39.935887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b716f8c-e393-4e45-b007-08f1816d0e1f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:39.936171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b716f8c-e393-4e45-b007-08f1816d0e1f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:39.936441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b716f8c-e393-4e45-b007-08f1816d0e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:39.937193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b716f8c-e393-4e45-b007-08f1816d0e1f None admin] RBAC: Adding query filter params (group.id=1198d79329074d26a5991f4153c0913c, role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=801abb9eafc94aef9a541cb855fb81cd) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:39.937506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b716f8c-e393-4e45-b007-08f1816d0e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:39.939286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b716f8c-e393-4e45-b007-08f1816d0e1f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:39.950457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 173/370] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/role_assignments?group.id=1198d79329074d26a5991f4153c0913c&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:39.951654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b716f8c-e393-4e45-b007-08f1816d0e1f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:39.970895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b77159-8a26-472a-b996-c4a78792cc1c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:40.013328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602b9d3-b7b3-433c-993c-69ffab27c7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:40.014464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602b9d3-b7b3-433c-993c-69ffab27c7c4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:40.014946 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602b9d3-b7b3-433c-993c-69ffab27c7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:40.015432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602b9d3-b7b3-433c-993c-69ffab27c7c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:40.016528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602b9d3-b7b3-433c-993c-69ffab27c7c4 None admin] RBAC: Adding query filter params (role.id=63d2a6efa7784a7eab9594616b3e8978, user.id=73ec0a11c5a44010a3f1efee1f76d16f, scope.system=all) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:40.017141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602b9d3-b7b3-433c-993c-69ffab27c7c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:40.018997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602b9d3-b7b3-433c-993c-69ffab27c7c4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:40.034554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 198/371] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/role_assignments?user.id=73ec0a11c5a44010a3f1efee1f76d16f&scope.system=all&role.id=63d2a6efa7784a7eab9594616b3e8978 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:40.036039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602b9d3-b7b3-433c-993c-69ffab27c7c4 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:40.076960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b716f8c-e393-4e45-b007-08f1816d0e1f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:40.078849 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:40.097573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:40.115218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454316af-ef8c-426a-b0f3-ba5c9ffcc4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:40.115957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454316af-ef8c-426a-b0f3-ba5c9ffcc4d2 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:40.116267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454316af-ef8c-426a-b0f3-ba5c9ffcc4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:40.116551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454316af-ef8c-426a-b0f3-ba5c9ffcc4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:40.117539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454316af-ef8c-426a-b0f3-ba5c9ffcc4d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:40.117539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454316af-ef8c-426a-b0f3-ba5c9ffcc4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:40.118996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454316af-ef8c-426a-b0f3-ba5c9ffcc4d2 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:40.129684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 174/372] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:40.135380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454316af-ef8c-426a-b0f3-ba5c9ffcc4d2 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:40.166009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602b9d3-b7b3-433c-993c-69ffab27c7c4 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:40.228891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d5b37-9fc1-4f95-bab1-43c794e6c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:40.230718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d5b37-9fc1-4f95-bab1-43c794e6c935 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:40.231236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d5b37-9fc1-4f95-bab1-43c794e6c935 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:40.231794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d5b37-9fc1-4f95-bab1-43c794e6c935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:40.253846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454316af-ef8c-426a-b0f3-ba5c9ffcc4d2 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:40.289915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48ecc0-9532-42ad-9ed0-f3153e9d8c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:40.289915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48ecc0-9532-42ad-9ed0-f3153e9d8c7a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:40.289915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48ecc0-9532-42ad-9ed0-f3153e9d8c7a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:40.289915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48ecc0-9532-42ad-9ed0-f3153e9d8c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:40.325756 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:40.342579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d5b37-9fc1-4f95-bab1-43c794e6c935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:40.344450 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d5b37-9fc1-4f95-bab1-43c794e6c935 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:40.369236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a48ecc0-9532-42ad-9ed0-f3153e9d8c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:40.370953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a48ecc0-9532-42ad-9ed0-f3153e9d8c7a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:40.402627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:40.459119 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 199/373] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:40.464877 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d5b37-9fc1-4f95-bab1-43c794e6c935 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:40.486919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 175/374] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:40.492714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48ecc0-9532-42ad-9ed0-f3153e9d8c7a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:40.563872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:40.575526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d5b37-9fc1-4f95-bab1-43c794e6c935 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:40.611605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48ecc0-9532-42ad-9ed0-f3153e9d8c7a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:40.618989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ca6f6-3e9b-403b-9d3a-5bd403e4e8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:40.619613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ca6f6-3e9b-403b-9d3a-5bd403e4e8d5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:40.622428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ca6f6-3e9b-403b-9d3a-5bd403e4e8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:40.622708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ca6f6-3e9b-403b-9d3a-5bd403e4e8d5 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:40.624486 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ca6f6-3e9b-403b-9d3a-5bd403e4e8d5 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:40.626064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ca6f6-3e9b-403b-9d3a-5bd403e4e8d5 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:40.628142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 200/375] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:40 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:40.633280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ca6f6-3e9b-403b-9d3a-5bd403e4e8d5 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:40.655711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3d9b6-6ebb-437b-a1f3-ce3c797dd319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:40.656639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:40.656772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3d9b6-6ebb-437b-a1f3-ce3c797dd319 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:40.657704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3d9b6-6ebb-437b-a1f3-ce3c797dd319 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:40.658008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3d9b6-6ebb-437b-a1f3-ce3c797dd319 None admin] PATH_INFO: `/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:40.659222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3d9b6-6ebb-437b-a1f3-ce3c797dd319 None admin] RBAC: Authorizing `identity:get_project(project_id=c0b6d65db9ee4a2cb85f51ab2f1fbc92)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:40.660742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3d9b6-6ebb-437b-a1f3-ce3c797dd319 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:40.669024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 176/376] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:40 2022] GET /identity/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:40.671467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3d9b6-6ebb-437b-a1f3-ce3c797dd319 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:40.752249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ca6f6-3e9b-403b-9d3a-5bd403e4e8d5 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:40.794654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9552dd-e45b-4ef1-b81f-6d1f78c883ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:40.795286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9552dd-e45b-4ef1-b81f-6d1f78c883ea None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:40.795559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9552dd-e45b-4ef1-b81f-6d1f78c883ea None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:40.795826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9552dd-e45b-4ef1-b81f-6d1f78c883ea None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:40.797523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9552dd-e45b-4ef1-b81f-6d1f78c883ea None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:40.799041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9552dd-e45b-4ef1-b81f-6d1f78c883ea None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:40.800950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 201/377] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:40 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:40.806655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9552dd-e45b-4ef1-b81f-6d1f78c883ea None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:40.847100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3d9b6-6ebb-437b-a1f3-ce3c797dd319 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:40.902575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:40.913340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26f5ef-b904-4eb7-a4a3-97662f51efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:40.913701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:40.914911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26f5ef-b904-4eb7-a4a3-97662f51efcf None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:40.915192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26f5ef-b904-4eb7-a4a3-97662f51efcf None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:40.915460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26f5ef-b904-4eb7-a4a3-97662f51efcf None admin] PATH_INFO: `/v3/groups/2926a97c5f2c4efb95fd77e4efd8210a` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:40.929555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9552dd-e45b-4ef1-b81f-6d1f78c883ea None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:40.931307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26f5ef-b904-4eb7-a4a3-97662f51efcf None admin] RBAC: Authorizing `identity:get_group(group_id=2926a97c5f2c4efb95fd77e4efd8210a)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:40.933436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26f5ef-b904-4eb7-a4a3-97662f51efcf None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:40.935551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 177/378] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:40 2022] GET /identity/v3/groups/2926a97c5f2c4efb95fd77e4efd8210a => generated 229 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:40.941167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26f5ef-b904-4eb7-a4a3-97662f51efcf None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:40.970301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256dea8-e9a9-4585-a3b0-ad33657400f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:40.970910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256dea8-e9a9-4585-a3b0-ad33657400f6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:40.971185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256dea8-e9a9-4585-a3b0-ad33657400f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:40.971791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256dea8-e9a9-4585-a3b0-ad33657400f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:40.972210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256dea8-e9a9-4585-a3b0-ad33657400f6 None admin] RBAC: Adding query filter params (group.id=1198d79329074d26a5991f4153c0913c, role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=801abb9eafc94aef9a541cb855fb81cd) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:40.972513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256dea8-e9a9-4585-a3b0-ad33657400f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:40.974165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256dea8-e9a9-4585-a3b0-ad33657400f6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:40.981934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 202/379] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:40 2022] GET /identity/v3/role_assignments?group.id=1198d79329074d26a5991f4153c0913c&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:41.094015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26f5ef-b904-4eb7-a4a3-97662f51efcf None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:41.137244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb877a3c-1a19-4b27-8f0c-18f13a7e12ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:41.137803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb877a3c-1a19-4b27-8f0c-18f13a7e12ea None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:41.138099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb877a3c-1a19-4b27-8f0c-18f13a7e12ea None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:41.138374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb877a3c-1a19-4b27-8f0c-18f13a7e12ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:41.139143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb877a3c-1a19-4b27-8f0c-18f13a7e12ea None admin] RBAC: Adding query filter params (group.id=2926a97c5f2c4efb95fd77e4efd8210a, role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=c0b6d65db9ee4a2cb85f51ab2f1fbc92) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:41.139455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb877a3c-1a19-4b27-8f0c-18f13a7e12ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:41.140920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb877a3c-1a19-4b27-8f0c-18f13a7e12ea None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:41.148781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 178/380] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:40 2022] GET /identity/v3/role_assignments?group.id=2926a97c5f2c4efb95fd77e4efd8210a&scope.project.id=c0b6d65db9ee4a2cb85f51ab2f1fbc92&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:41.365518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5cd7b-e420-4b7c-879c-daa03de19b5d None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:41.365908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5cd7b-e420-4b7c-879c-daa03de19b5d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:41.366084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5cd7b-e420-4b7c-879c-daa03de19b5d None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:41.367548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 203/381] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:41.372973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e4024-9999-4264-a42b-9578c078df40 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:41.373517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e4024-9999-4264-a42b-9578c078df40 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:41.374045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e4024-9999-4264-a42b-9578c078df40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:41.393030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b94e4024-9999-4264-a42b-9578c078df40 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:41.394581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b94e4024-9999-4264-a42b-9578c078df40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:41.466669 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:41.481059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 179/382] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:41.517399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5cd7b-e420-4b7c-879c-daa03de19b5d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:41.677632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5cd7b-e420-4b7c-879c-daa03de19b5d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:41.720251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8a8e1-0d1d-4696-a9a9-24c232e46149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:41.720886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8a8e1-0d1d-4696-a9a9-24c232e46149 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:41.721162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8a8e1-0d1d-4696-a9a9-24c232e46149 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:41.721432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8a8e1-0d1d-4696-a9a9-24c232e46149 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:41.735327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8a8e1-0d1d-4696-a9a9-24c232e46149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-0aa8a8e1-0d1d-4696-a9a9-24c232e46149 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:41.740170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:41.741061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:41.741867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:41.745903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 204/383] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:41.748101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e4024-9999-4264-a42b-9578c078df40 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:41.851618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e4024-9999-4264-a42b-9578c078df40 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:41.889446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff10bef-e651-49f4-b6dc-d57106e8fa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:41.890130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff10bef-e651-49f4-b6dc-d57106e8fa38 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:41.890406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff10bef-e651-49f4-b6dc-d57106e8fa38 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:41.890674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff10bef-e651-49f4-b6dc-d57106e8fa38 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:41.891341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff10bef-e651-49f4-b6dc-d57106e8fa38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:41.891646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff10bef-e651-49f4-b6dc-d57106e8fa38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:41.893033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff10bef-e651-49f4-b6dc-d57106e8fa38 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:41.901642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 180/384] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:41.907357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8a8e1-0d1d-4696-a9a9-24c232e46149 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:41.921445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:42.022532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8a8e1-0d1d-4696-a9a9-24c232e46149 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:42.063671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb7ae0-c6d3-4b11-9fe7-684a34b5cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:42.064298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb7ae0-c6d3-4b11-9fe7-684a34b5cb17 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:42.064577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb7ae0-c6d3-4b11-9fe7-684a34b5cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:42.064841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb7ae0-c6d3-4b11-9fe7-684a34b5cb17 None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:42.065971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb7ae0-c6d3-4b11-9fe7-684a34b5cb17 None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:42.067413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb7ae0-c6d3-4b11-9fe7-684a34b5cb17 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:42.073503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 205/385] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:41 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:42.075112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff10bef-e651-49f4-b6dc-d57106e8fa38 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:42.128444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:42.176234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff10bef-e651-49f4-b6dc-d57106e8fa38 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:42.210684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a195f-89b0-4db8-8421-94889293e75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:42.211313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a195f-89b0-4db8-8421-94889293e75b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:42.211586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a195f-89b0-4db8-8421-94889293e75b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:42.211852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a195f-89b0-4db8-8421-94889293e75b None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:42.212955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a195f-89b0-4db8-8421-94889293e75b None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:42.214972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a195f-89b0-4db8-8421-94889293e75b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:42.216590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 181/386] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:42 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:42.222664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb7ae0-c6d3-4b11-9fe7-684a34b5cb17 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:42.328976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:42.351516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb7ae0-c6d3-4b11-9fe7-684a34b5cb17 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:42.387575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480864e-e6a1-4d92-9d09-b5d545a53653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:42.388596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480864e-e6a1-4d92-9d09-b5d545a53653 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:42.389092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480864e-e6a1-4d92-9d09-b5d545a53653 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:42.389571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480864e-e6a1-4d92-9d09-b5d545a53653 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/groups/1198d79329074d26a5991f4153c0913c/roles/d7d75f4c26eb44689ec71efb8b63e534` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:42.392001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480864e-e6a1-4d92-9d09-b5d545a53653 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5bf4929d4984d40a41eaf5bd240ffd0, group_id=1198d79329074d26a5991f4153c0913c, role_id=d7d75f4c26eb44689ec71efb8b63e534)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:42.393669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480864e-e6a1-4d92-9d09-b5d545a53653 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:42.472180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 206/387] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Aug 8 16:40:42 2022] PUT /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/groups/1198d79329074d26a5991f4153c0913c/roles/d7d75f4c26eb44689ec71efb8b63e534 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:43.039942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa330f-7340-4cc8-9cd7-343cbc64e789 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.040217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa330f-7340-4cc8-9cd7-343cbc64e789 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.040727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa330f-7340-4cc8-9cd7-343cbc64e789 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.042623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 182/388] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:43.047619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed259cbc-ef73-4511-9881-0c3eb054406d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.048462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed259cbc-ef73-4511-9881-0c3eb054406d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.048889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed259cbc-ef73-4511-9881-0c3eb054406d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.055078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fecdd-82d1-42d8-9e3b-068a59debba2 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.055367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fecdd-82d1-42d8-9e3b-068a59debba2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.055644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fecdd-82d1-42d8-9e3b-068a59debba2 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.057809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 183/389] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:43.062814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddcbf6-767f-4386-942e-16f571899354 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.063094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddcbf6-767f-4386-942e-16f571899354 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.063370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddcbf6-767f-4386-942e-16f571899354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.070853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ed259cbc-ef73-4511-9881-0c3eb054406d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:43.072377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ed259cbc-ef73-4511-9881-0c3eb054406d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:43.098424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d8ddcbf6-767f-4386-942e-16f571899354 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:43.101233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d8ddcbf6-767f-4386-942e-16f571899354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:43.188047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 184/390] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:43.192218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 207/391] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:43.222415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddcbf6-767f-4386-942e-16f571899354 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:43.230386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed259cbc-ef73-4511-9881-0c3eb054406d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:43.289644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:43.356467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:43.384253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddcbf6-767f-4386-942e-16f571899354 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:43.423742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91594f44-0322-42eb-b665-64d3121cb2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:43.424368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91594f44-0322-42eb-b665-64d3121cb2bc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.424643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91594f44-0322-42eb-b665-64d3121cb2bc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.424907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91594f44-0322-42eb-b665-64d3121cb2bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.434246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed259cbc-ef73-4511-9881-0c3eb054406d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:43.446917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91594f44-0322-42eb-b665-64d3121cb2bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-91594f44-0322-42eb-b665-64d3121cb2bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:43.452623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:43.453550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:43.454416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:43.457820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 185/392] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:43.460805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91594f44-0322-42eb-b665-64d3121cb2bc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:43.502965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c38533-096d-4b79-84d8-f0809b8aefe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:43.503597 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c38533-096d-4b79-84d8-f0809b8aefe9 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.503898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c38533-096d-4b79-84d8-f0809b8aefe9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.504211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c38533-096d-4b79-84d8-f0809b8aefe9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.524391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c38533-096d-4b79-84d8-f0809b8aefe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-d2c38533-096d-4b79-84d8-f0809b8aefe9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:43.530620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:43.531528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:43.532321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:43.542585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 208/393] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:43.548170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c38533-096d-4b79-84d8-f0809b8aefe9 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:43.595263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91594f44-0322-42eb-b665-64d3121cb2bc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:43.607485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:43.631557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290fd66-9843-4a71-bbe3-85b6de272fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:43.632194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290fd66-9843-4a71-bbe3-85b6de272fe6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.632472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290fd66-9843-4a71-bbe3-85b6de272fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.632738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290fd66-9843-4a71-bbe3-85b6de272fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.633404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290fd66-9843-4a71-bbe3-85b6de272fe6 None admin] RBAC: Adding query filter params (name=member) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:43.633703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290fd66-9843-4a71-bbe3-85b6de272fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:43.635138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290fd66-9843-4a71-bbe3-85b6de272fe6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:43.649583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 186/394] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:43.649583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519744-99a8-45e7-a935-5f59da239dcf None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.649583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519744-99a8-45e7-a935-5f59da239dcf None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.649583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe519744-99a8-45e7-a935-5f59da239dcf None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.653210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 187/395] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:43.654592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8459-8586-4486-bd5e-e2ed1d5cbd77 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.654989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8459-8586-4486-bd5e-e2ed1d5cbd77 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.655261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8459-8586-4486-bd5e-e2ed1d5cbd77 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.656747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 188/396] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:43.657344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac8459-8586-4486-bd5e-e2ed1d5cbd77 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:43.674263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c38533-096d-4b79-84d8-f0809b8aefe9 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:43.702944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:43.711663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343f377-d9e9-4f39-a78d-3f82ca975adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:43.712267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343f377-d9e9-4f39-a78d-3f82ca975adf None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.712600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343f377-d9e9-4f39-a78d-3f82ca975adf None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.717166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343f377-d9e9-4f39-a78d-3f82ca975adf None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.717166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343f377-d9e9-4f39-a78d-3f82ca975adf None admin] RBAC: Adding query filter params (name=member) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:43.717166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343f377-d9e9-4f39-a78d-3f82ca975adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:43.717166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343f377-d9e9-4f39-a78d-3f82ca975adf None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:43.725505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 209/397] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:43.726757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80285204-9794-4525-a150-58947a3c31f1 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.727042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80285204-9794-4525-a150-58947a3c31f1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.727310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80285204-9794-4525-a150-58947a3c31f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.746420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-80285204-9794-4525-a150-58947a3c31f1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:43.747599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-80285204-9794-4525-a150-58947a3c31f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:43.770780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac8459-8586-4486-bd5e-e2ed1d5cbd77 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:43.807556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a9f8b-873c-4fe5-8837-33a55c346b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:43.808188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a9f8b-873c-4fe5-8837-33a55c346b69 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.808469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a9f8b-873c-4fe5-8837-33a55c346b69 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.808739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a9f8b-873c-4fe5-8837-33a55c346b69 None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.809982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a9f8b-873c-4fe5-8837-33a55c346b69 None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:43.811736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a9f8b-873c-4fe5-8837-33a55c346b69 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:43.817342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 189/398] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:43 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:43.817342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77288e-b139-4e64-a524-bee1ce7e989f None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.817342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77288e-b139-4e64-a524-bee1ce7e989f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.817342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77288e-b139-4e64-a524-bee1ce7e989f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.821949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 210/399] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:43.829943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80285204-9794-4525-a150-58947a3c31f1 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:43.843542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-9b77288e-b139-4e64-a524-bee1ce7e989f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:43.844843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9b77288e-b139-4e64-a524-bee1ce7e989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:43.882989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:43.917318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 190/400] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:43.918546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77288e-b139-4e64-a524-bee1ce7e989f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:43.941293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80285204-9794-4525-a150-58947a3c31f1 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:43.964769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:43.980691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d76ad-a0fd-4d89-9406-993993e09c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:43.981333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d76ad-a0fd-4d89-9406-993993e09c6c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:43.981612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d76ad-a0fd-4d89-9406-993993e09c6c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:43.982025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d76ad-a0fd-4d89-9406-993993e09c6c None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:43.983169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d76ad-a0fd-4d89-9406-993993e09c6c None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:43.984618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d76ad-a0fd-4d89-9406-993993e09c6c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:43.990776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 211/401] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:43 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:43.990776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d76ad-a0fd-4d89-9406-993993e09c6c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.041386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77288e-b139-4e64-a524-bee1ce7e989f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.078764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c556aa-3b3c-4904-95eb-f4e975caa2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.079794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c556aa-3b3c-4904-95eb-f4e975caa2e5 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.080328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c556aa-3b3c-4904-95eb-f4e975caa2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.080789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c556aa-3b3c-4904-95eb-f4e975caa2e5 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.082198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c556aa-3b3c-4904-95eb-f4e975caa2e5 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:44.083919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c556aa-3b3c-4904-95eb-f4e975caa2e5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:44.086268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 191/402] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:43 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:44.087178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c556aa-3b3c-4904-95eb-f4e975caa2e5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.093433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d76ad-a0fd-4d89-9406-993993e09c6c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.129417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e216189-0878-4705-b834-21a6e8908e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.129602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:44.130698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e216189-0878-4705-b834-21a6e8908e32 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.131145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e216189-0878-4705-b834-21a6e8908e32 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.131569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e216189-0878-4705-b834-21a6e8908e32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.141956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e216189-0878-4705-b834-21a6e8908e32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-2e216189-0878-4705-b834-21a6e8908e32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:44.147277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:44.151693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 212/403] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:44.152468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e216189-0878-4705-b834-21a6e8908e32 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.194963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c556aa-3b3c-4904-95eb-f4e975caa2e5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.202732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:44.230412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb7488-1322-4b75-a8d6-40a7f93d94fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.231038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb7488-1322-4b75-a8d6-40a7f93d94fe None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.231319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb7488-1322-4b75-a8d6-40a7f93d94fe None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.231586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb7488-1322-4b75-a8d6-40a7f93d94fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.241819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb7488-1322-4b75-a8d6-40a7f93d94fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-16eb7488-1322-4b75-a8d6-40a7f93d94fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:44.247080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:44.248169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:44.249052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 192/404] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:40:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:44.249726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb7488-1322-4b75-a8d6-40a7f93d94fe None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.260936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e216189-0878-4705-b834-21a6e8908e32 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.298350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dc5b7-28de-4733-9e8f-f9514bcadade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.299303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dc5b7-28de-4733-9e8f-f9514bcadade None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.299725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dc5b7-28de-4733-9e8f-f9514bcadade None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.300118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dc5b7-28de-4733-9e8f-f9514bcadade None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.301409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dc5b7-28de-4733-9e8f-f9514bcadade None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:44.303570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dc5b7-28de-4733-9e8f-f9514bcadade None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:44.311023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 213/405] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:44 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:44.311023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dc5b7-28de-4733-9e8f-f9514bcadade None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.357866 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:44.362331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb7488-1322-4b75-a8d6-40a7f93d94fe None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.397950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5323b-de17-4c9b-9027-d4fa5c4385b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.398528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5323b-de17-4c9b-9027-d4fa5c4385b5 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.398801 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5323b-de17-4c9b-9027-d4fa5c4385b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.399064 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5323b-de17-4c9b-9027-d4fa5c4385b5 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd/groups/1198d79329074d26a5991f4153c0913c/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.401590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5323b-de17-4c9b-9027-d4fa5c4385b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=801abb9eafc94aef9a541cb855fb81cd, group_id=1198d79329074d26a5991f4153c0913c, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:44.403076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5323b-de17-4c9b-9027-d4fa5c4385b5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:44.416726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dc5b7-28de-4733-9e8f-f9514bcadade None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.432326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:44.451008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b380ff1-a46f-46a1-80d8-08c3bb2756d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.451619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b380ff1-a46f-46a1-80d8-08c3bb2756d3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.451897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b380ff1-a46f-46a1-80d8-08c3bb2756d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.452168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b380ff1-a46f-46a1-80d8-08c3bb2756d3 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.452838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b380ff1-a46f-46a1-80d8-08c3bb2756d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:44.453145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b380ff1-a46f-46a1-80d8-08c3bb2756d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:44.454576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b380ff1-a46f-46a1-80d8-08c3bb2756d3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:44.457680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 193/406] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Aug 8 16:40:44 2022] PUT /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd/groups/1198d79329074d26a5991f4153c0913c/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:44.458722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5323b-de17-4c9b-9027-d4fa5c4385b5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.465827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 214/407] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:44.471294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b380ff1-a46f-46a1-80d8-08c3bb2756d3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.613669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b380ff1-a46f-46a1-80d8-08c3bb2756d3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.615286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5323b-de17-4c9b-9027-d4fa5c4385b5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.654579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84824b2c-7e31-4460-9107-7bf971a5576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.655510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84824b2c-7e31-4460-9107-7bf971a5576c None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.655936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84824b2c-7e31-4460-9107-7bf971a5576c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.656215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84824b2c-7e31-4460-9107-7bf971a5576c None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/groups/1198d79329074d26a5991f4153c0913c/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.658299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84824b2c-7e31-4460-9107-7bf971a5576c None admin] RBAC: Authorizing `identity:create_grant(project_id=b5bf4929d4984d40a41eaf5bd240ffd0, group_id=1198d79329074d26a5991f4153c0913c, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:44.659987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84824b2c-7e31-4460-9107-7bf971a5576c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:44.670162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f0fd5-2183-4d9b-b3f4-d0a1a5fac024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.670858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f0fd5-2183-4d9b-b3f4-d0a1a5fac024 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.671211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f0fd5-2183-4d9b-b3f4-d0a1a5fac024 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.671571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f0fd5-2183-4d9b-b3f4-d0a1a5fac024 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.672306 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f0fd5-2183-4d9b-b3f4-d0a1a5fac024 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:44.672676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f0fd5-2183-4d9b-b3f4-d0a1a5fac024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:44.674172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f0fd5-2183-4d9b-b3f4-d0a1a5fac024 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:44.683270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 194/408] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:44.684969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f0fd5-2183-4d9b-b3f4-d0a1a5fac024 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.707456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:44.737582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 215/409] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Aug 8 16:40:44 2022] PUT /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0/groups/1198d79329074d26a5991f4153c0913c/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:44.738810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84824b2c-7e31-4460-9107-7bf971a5576c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.795797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:44.865497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84824b2c-7e31-4460-9107-7bf971a5576c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.872866 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f0fd5-2183-4d9b-b3f4-d0a1a5fac024 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:44.909491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3136ff2-b54e-43a9-a2d8-d4ba81e808bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.910690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3136ff2-b54e-43a9-a2d8-d4ba81e808bc None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.911547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3136ff2-b54e-43a9-a2d8-d4ba81e808bc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.912075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3136ff2-b54e-43a9-a2d8-d4ba81e808bc None admin] PATH_INFO: `/v3/groups/2926a97c5f2c4efb95fd77e4efd8210a` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.913496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3136ff2-b54e-43a9-a2d8-d4ba81e808bc None admin] RBAC: Authorizing `identity:get_group(group_id=2926a97c5f2c4efb95fd77e4efd8210a)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:44.915988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3136ff2-b54e-43a9-a2d8-d4ba81e808bc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:44.919088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 216/410] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:44 2022] GET /identity/v3/groups/2926a97c5f2c4efb95fd77e4efd8210a => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:44.921081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8eca0-99f7-46cb-9546-ad7a1711c76a None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.921538 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8eca0-99f7-46cb-9546-ad7a1711c76a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.922356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8eca0-99f7-46cb-9546-ad7a1711c76a None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.924282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 217/411] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:44.925201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8eca0-99f7-46cb-9546-ad7a1711c76a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:44.937692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e09c91-c521-4dea-a71d-17ea0beda3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:44.939156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e09c91-c521-4dea-a71d-17ea0beda3f2 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.939462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e09c91-c521-4dea-a71d-17ea0beda3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.939909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e09c91-c521-4dea-a71d-17ea0beda3f2 None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.941689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e09c91-c521-4dea-a71d-17ea0beda3f2 None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:44.943334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e09c91-c521-4dea-a71d-17ea0beda3f2 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:44.945114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 195/412] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:44 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:44.946440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22ed5c-2238-4f6a-b4ac-deff25c0c2ae None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:44.946795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22ed5c-2238-4f6a-b4ac-deff25c0c2ae None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:44.947215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22ed5c-2238-4f6a-b4ac-deff25c0c2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:44.969472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3e22ed5c-2238-4f6a-b4ac-deff25c0c2ae None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:44.971388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3e22ed5c-2238-4f6a-b4ac-deff25c0c2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:45.053607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 196/413] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:45.054703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22ed5c-2238-4f6a-b4ac-deff25c0c2ae None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:45.071917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8eca0-99f7-46cb-9546-ad7a1711c76a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:45.075044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:45.086460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:45.124585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e64c4-cf0e-4a8c-80ef-7aa250e4e675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:45.125198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e64c4-cf0e-4a8c-80ef-7aa250e4e675 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:45.125486 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e64c4-cf0e-4a8c-80ef-7aa250e4e675 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:45.125759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e64c4-cf0e-4a8c-80ef-7aa250e4e675 None admin] PATH_INFO: `/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:45.126916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e64c4-cf0e-4a8c-80ef-7aa250e4e675 None admin] RBAC: Authorizing `identity:get_project(project_id=c0b6d65db9ee4a2cb85f51ab2f1fbc92)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:45.128479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e64c4-cf0e-4a8c-80ef-7aa250e4e675 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:45.130535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 218/414] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:44 2022] GET /identity/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92 => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:45.131318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e64c4-cf0e-4a8c-80ef-7aa250e4e675 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:45.164212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22ed5c-2238-4f6a-b4ac-deff25c0c2ae None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:45.200495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c01282-fe49-4155-86a1-e54177e38de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:45.201188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c01282-fe49-4155-86a1-e54177e38de3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:45.201470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c01282-fe49-4155-86a1-e54177e38de3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:45.201743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c01282-fe49-4155-86a1-e54177e38de3 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:45.202939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c01282-fe49-4155-86a1-e54177e38de3 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:45.204463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c01282-fe49-4155-86a1-e54177e38de3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:45.211051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 197/415] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:45 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:45.211891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c01282-fe49-4155-86a1-e54177e38de3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:45.236992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e64c4-cf0e-4a8c-80ef-7aa250e4e675 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:45.273469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73edeee-5c22-4653-82be-c809154479d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:45.274336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73edeee-5c22-4653-82be-c809154479d5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:45.274691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73edeee-5c22-4653-82be-c809154479d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:45.274969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73edeee-5c22-4653-82be-c809154479d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:45.286394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73edeee-5c22-4653-82be-c809154479d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-e73edeee-5c22-4653-82be-c809154479d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:45.292224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:45.300824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 219/416] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:45.302874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73edeee-5c22-4653-82be-c809154479d5 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:45.318895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c01282-fe49-4155-86a1-e54177e38de3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:45.334734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:45.342221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:45.356944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d356d00-0e0b-405e-aadc-f42bc9a46047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:45.357908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d356d00-0e0b-405e-aadc-f42bc9a46047 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:45.358329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d356d00-0e0b-405e-aadc-f42bc9a46047 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:45.358760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d356d00-0e0b-405e-aadc-f42bc9a46047 None admin] PATH_INFO: `/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92/groups/2926a97c5f2c4efb95fd77e4efd8210a/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:45.361250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d356d00-0e0b-405e-aadc-f42bc9a46047 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0b6d65db9ee4a2cb85f51ab2f1fbc92, group_id=2926a97c5f2c4efb95fd77e4efd8210a, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:45.362947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d356d00-0e0b-405e-aadc-f42bc9a46047 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:45.405668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73edeee-5c22-4653-82be-c809154479d5 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:45.440828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 198/417] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Aug 8 16:40:45 2022] PUT /identity/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92/groups/2926a97c5f2c4efb95fd77e4efd8210a/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:45.440828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d356d00-0e0b-405e-aadc-f42bc9a46047 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:45.452391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e80782-1b11-49c5-bb02-a66e574979fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:45.454624 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e80782-1b11-49c5-bb02-a66e574979fb None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:45.455474 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e80782-1b11-49c5-bb02-a66e574979fb None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:45.455759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e80782-1b11-49c5-bb02-a66e574979fb None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd/groups/1198d79329074d26a5991f4153c0913c/roles/d7d75f4c26eb44689ec71efb8b63e534` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:45.461746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e80782-1b11-49c5-bb02-a66e574979fb None admin] RBAC: Authorizing `identity:create_grant(project_id=801abb9eafc94aef9a541cb855fb81cd, group_id=1198d79329074d26a5991f4153c0913c, role_id=d7d75f4c26eb44689ec71efb8b63e534)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:45.464189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e80782-1b11-49c5-bb02-a66e574979fb None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:45.540999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 220/418] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Aug 8 16:40:45 2022] PUT /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd/groups/1198d79329074d26a5991f4153c0913c/roles/d7d75f4c26eb44689ec71efb8b63e534 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:40:45.587076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d356d00-0e0b-405e-aadc-f42bc9a46047 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:45.626581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29f82-88ea-4971-89b1-ffd8e6888726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:45.627209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29f82-88ea-4971-89b1-ffd8e6888726 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:45.627492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29f82-88ea-4971-89b1-ffd8e6888726 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:45.627766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29f82-88ea-4971-89b1-ffd8e6888726 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:45.628443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed29f82-88ea-4971-89b1-ffd8e6888726 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:45.628747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed29f82-88ea-4971-89b1-ffd8e6888726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:45.631741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed29f82-88ea-4971-89b1-ffd8e6888726 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:45.649539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 199/419] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:45.656532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e80782-1b11-49c5-bb02-a66e574979fb None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:45.774690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:45.783238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e80782-1b11-49c5-bb02-a66e574979fb None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:45.826425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807d6c-79eb-4ad7-810d-e988a81647ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:45.827039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807d6c-79eb-4ad7-810d-e988a81647ad None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:45.827316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807d6c-79eb-4ad7-810d-e988a81647ad None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:45.827587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807d6c-79eb-4ad7-810d-e988a81647ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:45.899615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35807d6c-79eb-4ad7-810d-e988a81647ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:45.901108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35807d6c-79eb-4ad7-810d-e988a81647ad None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:46.028005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:46.036061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 221/420] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:46.041571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29f82-88ea-4971-89b1-ffd8e6888726 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:46.063849 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:46.195450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29f82-88ea-4971-89b1-ffd8e6888726 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:46.248851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f1d2a-f4e4-4a69-84f5-eae1768aa270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:46.249496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f1d2a-f4e4-4a69-84f5-eae1768aa270 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:46.249771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f1d2a-f4e4-4a69-84f5-eae1768aa270 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:46.250063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f1d2a-f4e4-4a69-84f5-eae1768aa270 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:46.251202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f1d2a-f4e4-4a69-84f5-eae1768aa270 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:46.252629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f1d2a-f4e4-4a69-84f5-eae1768aa270 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:46.254606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 200/421] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:46 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:46.259674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807d6c-79eb-4ad7-810d-e988a81647ad None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:46.388043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807d6c-79eb-4ad7-810d-e988a81647ad None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:46.425243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7920fe-5efb-4780-8b4d-09e4d4a5ac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:46.425893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7920fe-5efb-4780-8b4d-09e4d4a5ac4f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:46.426172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7920fe-5efb-4780-8b4d-09e4d4a5ac4f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:46.426440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7920fe-5efb-4780-8b4d-09e4d4a5ac4f None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:46.427537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7920fe-5efb-4780-8b4d-09e4d4a5ac4f None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:46.428982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7920fe-5efb-4780-8b4d-09e4d4a5ac4f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:46.430581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 222/422] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:46 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:46.436329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f1d2a-f4e4-4a69-84f5-eae1768aa270 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:46.538700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:46.567294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f1d2a-f4e4-4a69-84f5-eae1768aa270 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:46.625637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88df501-b31d-4f26-8ee7-cc527e321462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:46.626269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88df501-b31d-4f26-8ee7-cc527e321462 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:46.626546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88df501-b31d-4f26-8ee7-cc527e321462 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:46.626816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88df501-b31d-4f26-8ee7-cc527e321462 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:46.627540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88df501-b31d-4f26-8ee7-cc527e321462 None admin] RBAC: Adding query filter params (group.id=1198d79329074d26a5991f4153c0913c, role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:46.627846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88df501-b31d-4f26-8ee7-cc527e321462 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:46.629282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88df501-b31d-4f26-8ee7-cc527e321462 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:46.639563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 201/423] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:46 2022] GET /identity/v3/role_assignments?group.id=1198d79329074d26a5991f4153c0913c&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:46.668467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae62cd-407e-4970-95d3-9a80549402e6 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:46.668771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae62cd-407e-4970-95d3-9a80549402e6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:46.669019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae62cd-407e-4970-95d3-9a80549402e6 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:46.670749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 223/424] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:46.676015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ce2a-fa41-4f01-9448-d47ddd049813 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:46.676291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ce2a-fa41-4f01-9448-d47ddd049813 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:46.676564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ce2a-fa41-4f01-9448-d47ddd049813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:46.704141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4126ce2a-fa41-4f01-9448-d47ddd049813 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:46.705690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4126ce2a-fa41-4f01-9448-d47ddd049813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:46.802590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 202/425] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:46.827385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae62cd-407e-4970-95d3-9a80549402e6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:46.897857 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:46.939380 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae62cd-407e-4970-95d3-9a80549402e6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:46.981151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48957c52-22f0-4665-bbda-f573c5c916f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:46.981816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48957c52-22f0-4665-bbda-f573c5c916f0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:46.982158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48957c52-22f0-4665-bbda-f573c5c916f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:46.982430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48957c52-22f0-4665-bbda-f573c5c916f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:46.993235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48957c52-22f0-4665-bbda-f573c5c916f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-48957c52-22f0-4665-bbda-f573c5c916f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:46.999971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:47.000847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:47.001623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 224/426] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:47.007758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126ce2a-fa41-4f01-9448-d47ddd049813 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:47.102237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:47.134115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126ce2a-fa41-4f01-9448-d47ddd049813 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:47.171213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cee75-c2b4-4995-bf86-f8812cba1d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:47.171733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cee75-c2b4-4995-bf86-f8812cba1d24 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:47.172007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cee75-c2b4-4995-bf86-f8812cba1d24 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:47.172269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cee75-c2b4-4995-bf86-f8812cba1d24 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:47.172924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cee75-c2b4-4995-bf86-f8812cba1d24 None admin] RBAC: Adding query filter params (name=member) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:47.173216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cee75-c2b4-4995-bf86-f8812cba1d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:47.174602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cee75-c2b4-4995-bf86-f8812cba1d24 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:47.183551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 203/427] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:47.188772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48957c52-22f0-4665-bbda-f573c5c916f0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:47.250960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b5b80-263e-45cb-a6fc-0b7179c5a5fe None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:47.251915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b5b80-263e-45cb-a6fc-0b7179c5a5fe None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:47.252396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b5b80-263e-45cb-a6fc-0b7179c5a5fe None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:47.254210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 204/428] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:47.259204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1a2a2-d203-409c-8076-21d70a21e889 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:47.259695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1a2a2-d203-409c-8076-21d70a21e889 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:47.260727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1a2a2-d203-409c-8076-21d70a21e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:47.279332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4fb1a2a2-d203-409c-8076-21d70a21e889 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:47.280807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4fb1a2a2-d203-409c-8076-21d70a21e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:47.303630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48957c52-22f0-4665-bbda-f573c5c916f0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:47.330962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:47.343509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f0840-501f-4d4e-9583-baa39b9921b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:47.344369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f0840-501f-4d4e-9583-baa39b9921b6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:47.344847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f0840-501f-4d4e-9583-baa39b9921b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:47.345283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f0840-501f-4d4e-9583-baa39b9921b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:47.361402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 205/429] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:47.381357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1a2a2-d203-409c-8076-21d70a21e889 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:47.430347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f0840-501f-4d4e-9583-baa39b9921b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:47.432490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f0840-501f-4d4e-9583-baa39b9921b6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:47.449104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:47.493050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1a2a2-d203-409c-8076-21d70a21e889 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:47.532394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe714b8-7183-4550-aa42-ca94b0f6e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:47.533501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe714b8-7183-4550-aa42-ca94b0f6e399 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:47.534130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe714b8-7183-4550-aa42-ca94b0f6e399 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:47.534618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe714b8-7183-4550-aa42-ca94b0f6e399 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:47.546681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe714b8-7183-4550-aa42-ca94b0f6e399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-cfe714b8-7183-4550-aa42-ca94b0f6e399 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:47.552449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:47.553324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 16:40:47.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:47.561194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 225/430] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:47.561735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 206/431] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:47.562172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f0840-501f-4d4e-9583-baa39b9921b6 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:47.563208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe714b8-7183-4550-aa42-ca94b0f6e399 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:47.591210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:47.677083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f0840-501f-4d4e-9583-baa39b9921b6 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:47.687456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe714b8-7183-4550-aa42-ca94b0f6e399 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:47.714149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98616e8e-1d34-495c-b139-f6b62fbbfb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:47.715104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98616e8e-1d34-495c-b139-f6b62fbbfb84 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:47.715211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:47.715678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98616e8e-1d34-495c-b139-f6b62fbbfb84 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:47.716146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98616e8e-1d34-495c-b139-f6b62fbbfb84 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:47.717594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98616e8e-1d34-495c-b139-f6b62fbbfb84 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:47.719258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98616e8e-1d34-495c-b139-f6b62fbbfb84 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:47.721426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 226/432] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:47 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:47.726918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98616e8e-1d34-495c-b139-f6b62fbbfb84 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:47.732477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d688e2-b88c-4ce2-beee-3b7e16bb343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:47.734130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d688e2-b88c-4ce2-beee-3b7e16bb343a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:47.734485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d688e2-b88c-4ce2-beee-3b7e16bb343a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:47.734763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d688e2-b88c-4ce2-beee-3b7e16bb343a None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:47.735434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d688e2-b88c-4ce2-beee-3b7e16bb343a None admin] RBAC: Adding query filter params (name=member) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:47.735734 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d688e2-b88c-4ce2-beee-3b7e16bb343a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:47.737164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d688e2-b88c-4ce2-beee-3b7e16bb343a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:47.746804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 207/433] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:40:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:47.758553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d688e2-b88c-4ce2-beee-3b7e16bb343a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:47.845664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:47.859765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98616e8e-1d34-495c-b139-f6b62fbbfb84 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:47.884198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d688e2-b88c-4ce2-beee-3b7e16bb343a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:47.905846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870cc5d-c0f0-46e6-be07-f5144891997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:47.906773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870cc5d-c0f0-46e6-be07-f5144891997d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:47.907193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870cc5d-c0f0-46e6-be07-f5144891997d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:47.907657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870cc5d-c0f0-46e6-be07-f5144891997d None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:47.908948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870cc5d-c0f0-46e6-be07-f5144891997d None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:47.910559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870cc5d-c0f0-46e6-be07-f5144891997d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:47.917835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 227/434] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:47 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:47.918849 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870cc5d-c0f0-46e6-be07-f5144891997d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:47.923927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162364d5-b663-44ef-891d-df77d697e074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:47.924553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162364d5-b663-44ef-891d-df77d697e074 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:47.924837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162364d5-b663-44ef-891d-df77d697e074 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:47.925114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162364d5-b663-44ef-891d-df77d697e074 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:47.992588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:48.009710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162364d5-b663-44ef-891d-df77d697e074 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:48.011710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162364d5-b663-44ef-891d-df77d697e074 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:48.023786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870cc5d-c0f0-46e6-be07-f5144891997d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.067443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e6f8b-e67c-418b-b979-0644c891f0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.068307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e6f8b-e67c-418b-b979-0644c891f0b4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.068584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e6f8b-e67c-418b-b979-0644c891f0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.068907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e6f8b-e67c-418b-b979-0644c891f0b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.069816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e6f8b-e67c-418b-b979-0644c891f0b4 None admin] RBAC: Adding query filter params (group.id=1198d79329074d26a5991f4153c0913c, role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=801abb9eafc94aef9a541cb855fb81cd) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:48.070434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e6f8b-e67c-418b-b979-0644c891f0b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:48.072915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e6f8b-e67c-418b-b979-0644c891f0b4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:48.079302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 228/435] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:47 2022] GET /identity/v3/role_assignments?group.id=1198d79329074d26a5991f4153c0913c&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:48.080461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1716a2-c4de-4358-bb34-d80c81923cf2 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.080744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1716a2-c4de-4358-bb34-d80c81923cf2 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.081012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1716a2-c4de-4358-bb34-d80c81923cf2 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.085842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 229/436] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:48.085842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34ac17-33dd-48f8-96c6-9db899b8fa76 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.085842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34ac17-33dd-48f8-96c6-9db899b8fa76 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.085842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34ac17-33dd-48f8-96c6-9db899b8fa76 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.087325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 230/437] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:48.091990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffecb3-6c38-4c6f-9319-44e2c651f0ff None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.091990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffecb3-6c38-4c6f-9319-44e2c651f0ff None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.091990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffecb3-6c38-4c6f-9319-44e2c651f0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.109608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-abffecb3-6c38-4c6f-9319-44e2c651f0ff None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:48.110868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-abffecb3-6c38-4c6f-9319-44e2c651f0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:48.124759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 208/438] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:48.126055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86663b76-d5a4-4ede-823a-10c80b3658a2 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.126341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86663b76-d5a4-4ede-823a-10c80b3658a2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.126963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86663b76-d5a4-4ede-823a-10c80b3658a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.130212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:48.146279 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-86663b76-d5a4-4ede-823a-10c80b3658a2 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:48.147578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-86663b76-d5a4-4ede-823a-10c80b3658a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:48.184248 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 231/439] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:48.185514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffecb3-6c38-4c6f-9319-44e2c651f0ff None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:48.226040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 209/440] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:48.226742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86663b76-d5a4-4ede-823a-10c80b3658a2 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:48.243508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:48.312485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffecb3-6c38-4c6f-9319-44e2c651f0ff None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.341461 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86663b76-d5a4-4ede-823a-10c80b3658a2 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.346745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6475f5-57cd-4d83-a888-d995991ec2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.347370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6475f5-57cd-4d83-a888-d995991ec2d0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.347646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6475f5-57cd-4d83-a888-d995991ec2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.347914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6475f5-57cd-4d83-a888-d995991ec2d0 None admin] PATH_INFO: `/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.349054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6475f5-57cd-4d83-a888-d995991ec2d0 None admin] RBAC: Authorizing `identity:get_project(project_id=b5bf4929d4984d40a41eaf5bd240ffd0)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:48.350542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6475f5-57cd-4d83-a888-d995991ec2d0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:48.352472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 232/441] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:48 2022] GET /identity/v3/projects/b5bf4929d4984d40a41eaf5bd240ffd0 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:48.353151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6475f5-57cd-4d83-a888-d995991ec2d0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:48.375021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcbf37-a223-47e4-bebe-0139b34eecb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.375651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcbf37-a223-47e4-bebe-0139b34eecb1 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.375929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcbf37-a223-47e4-bebe-0139b34eecb1 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.376199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcbf37-a223-47e4-bebe-0139b34eecb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.377687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:48.386911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcbf37-a223-47e4-bebe-0139b34eecb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-3abcbf37-a223-47e4-bebe-0139b34eecb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:48.391775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:48.392875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:48.393891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 210/442] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:40:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:48.394663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcbf37-a223-47e4-bebe-0139b34eecb1 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:48.458330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6475f5-57cd-4d83-a888-d995991ec2d0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.466555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:48.490063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcbf37-a223-47e4-bebe-0139b34eecb1 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.491729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3535301-95b3-4bcf-aa74-6cdbf6766b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.492351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3535301-95b3-4bcf-aa74-6cdbf6766b7d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.492631 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3535301-95b3-4bcf-aa74-6cdbf6766b7d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.492979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3535301-95b3-4bcf-aa74-6cdbf6766b7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.503512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3535301-95b3-4bcf-aa74-6cdbf6766b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-d3535301-95b3-4bcf-aa74-6cdbf6766b7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:48.508130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:48.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:48.510470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 233/443] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:48.511109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3535301-95b3-4bcf-aa74-6cdbf6766b7d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:48.525166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244e67e-c1ba-4a30-9c3f-f2f14a64ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.525848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244e67e-c1ba-4a30-9c3f-f2f14a64ca52 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.526128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244e67e-c1ba-4a30-9c3f-f2f14a64ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.526399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244e67e-c1ba-4a30-9c3f-f2f14a64ca52 None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.527513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244e67e-c1ba-4a30-9c3f-f2f14a64ca52 None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:48.528948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244e67e-c1ba-4a30-9c3f-f2f14a64ca52 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:48.530547 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 211/444] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:48 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:48.534630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244e67e-c1ba-4a30-9c3f-f2f14a64ca52 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:48.595525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:48.614599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3535301-95b3-4bcf-aa74-6cdbf6766b7d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.638015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244e67e-c1ba-4a30-9c3f-f2f14a64ca52 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.649727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01298c97-fc79-4aa0-993e-02edb27dd119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.650460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01298c97-fc79-4aa0-993e-02edb27dd119 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.651224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01298c97-fc79-4aa0-993e-02edb27dd119 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.651493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01298c97-fc79-4aa0-993e-02edb27dd119 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.652153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01298c97-fc79-4aa0-993e-02edb27dd119 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:48.652451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01298c97-fc79-4aa0-993e-02edb27dd119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:48.653854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01298c97-fc79-4aa0-993e-02edb27dd119 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:48.661553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 234/445] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:48.662355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01298c97-fc79-4aa0-993e-02edb27dd119 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:48.671680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a264329-9e34-41f2-8788-0a0fd3428734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.672283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a264329-9e34-41f2-8788-0a0fd3428734 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.672559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a264329-9e34-41f2-8788-0a0fd3428734 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.672827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a264329-9e34-41f2-8788-0a0fd3428734 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.673483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a264329-9e34-41f2-8788-0a0fd3428734 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:48.673804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a264329-9e34-41f2-8788-0a0fd3428734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:48.675167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a264329-9e34-41f2-8788-0a0fd3428734 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:48.683017 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 212/446] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:48.683800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a264329-9e34-41f2-8788-0a0fd3428734 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:48.687970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:48.761425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01298c97-fc79-4aa0-993e-02edb27dd119 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.784444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a264329-9e34-41f2-8788-0a0fd3428734 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.795051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007893f-82d0-4330-a8c9-3b2c86ceac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.795656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007893f-82d0-4330-a8c9-3b2c86ceac1b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.795935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007893f-82d0-4330-a8c9-3b2c86ceac1b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.796205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007893f-82d0-4330-a8c9-3b2c86ceac1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.796926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007893f-82d0-4330-a8c9-3b2c86ceac1b None admin] RBAC: Adding query filter params (group.id=1198d79329074d26a5991f4153c0913c, role.id=d01bf1c8bfca4691affaddcd9487c751, scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:48.797230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007893f-82d0-4330-a8c9-3b2c86ceac1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:48.798695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007893f-82d0-4330-a8c9-3b2c86ceac1b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:48.817816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 235/447] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:48 2022] GET /identity/v3/role_assignments?group.id=1198d79329074d26a5991f4153c0913c&scope.project.id=b5bf4929d4984d40a41eaf5bd240ffd0&role.id=d01bf1c8bfca4691affaddcd9487c751 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:48.817816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007893f-82d0-4330-a8c9-3b2c86ceac1b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:48.817816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:48.833391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f71219-fd3c-4f7a-874b-acede558eabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.834063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f71219-fd3c-4f7a-874b-acede558eabc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.834345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f71219-fd3c-4f7a-874b-acede558eabc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.834617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f71219-fd3c-4f7a-874b-acede558eabc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:48.912621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f71219-fd3c-4f7a-874b-acede558eabc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:48.914063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f71219-fd3c-4f7a-874b-acede558eabc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:48.918376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:48.920755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007893f-82d0-4330-a8c9-3b2c86ceac1b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:48.960184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271ef33-032a-473b-8517-1984e046f121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:48.960826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271ef33-032a-473b-8517-1984e046f121 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:48.961322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271ef33-032a-473b-8517-1984e046f121 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:48.961435 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271ef33-032a-473b-8517-1984e046f121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:49.030657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 213/448] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:49.036124 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f71219-fd3c-4f7a-874b-acede558eabc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:49.037865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271ef33-032a-473b-8517-1984e046f121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:49.039322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271ef33-032a-473b-8517-1984e046f121 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:49.049929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:49.151706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f71219-fd3c-4f7a-874b-acede558eabc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:49.156399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 236/449] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:49.157041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:49.162051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271ef33-032a-473b-8517-1984e046f121 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:49.195627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e0d3a-8dc6-47df-84c6-dfcfa1ced657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:49.196532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e0d3a-8dc6-47df-84c6-dfcfa1ced657 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:49.197995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e0d3a-8dc6-47df-84c6-dfcfa1ced657 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:49.198276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e0d3a-8dc6-47df-84c6-dfcfa1ced657 None admin] PATH_INFO: `/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:49.199705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e0d3a-8dc6-47df-84c6-dfcfa1ced657 None admin] RBAC: Authorizing `identity:get_project(project_id=c0b6d65db9ee4a2cb85f51ab2f1fbc92)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:49.201825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e0d3a-8dc6-47df-84c6-dfcfa1ced657 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:49.202825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 214/450] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:49 2022] GET /identity/v3/projects/c0b6d65db9ee4a2cb85f51ab2f1fbc92 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:49.208199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e0d3a-8dc6-47df-84c6-dfcfa1ced657 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:49.285507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271ef33-032a-473b-8517-1984e046f121 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:49.293746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:49.324363 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343858a-68be-40e1-aabe-814ef7f45548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:49.324984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e0d3a-8dc6-47df-84c6-dfcfa1ced657 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:49.325065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343858a-68be-40e1-aabe-814ef7f45548 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:49.325266 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343858a-68be-40e1-aabe-814ef7f45548 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:49.325522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343858a-68be-40e1-aabe-814ef7f45548 None admin] PATH_INFO: `/v3/projects/801abb9eafc94aef9a541cb855fb81cd` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:49.326682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343858a-68be-40e1-aabe-814ef7f45548 None admin] RBAC: Authorizing `identity:get_project(project_id=801abb9eafc94aef9a541cb855fb81cd)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:49.328099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343858a-68be-40e1-aabe-814ef7f45548 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:49.329927 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 237/451] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:40:49 2022] GET /identity/v3/projects/801abb9eafc94aef9a541cb855fb81cd => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:49.335752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343858a-68be-40e1-aabe-814ef7f45548 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:49.370610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f81109-94b9-4d14-9120-ca0d49065fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:49.371245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f81109-94b9-4d14-9120-ca0d49065fcd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:49.371532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f81109-94b9-4d14-9120-ca0d49065fcd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:49.371803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f81109-94b9-4d14-9120-ca0d49065fcd None admin] PATH_INFO: `/v3/groups/2926a97c5f2c4efb95fd77e4efd8210a` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:49.372923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f81109-94b9-4d14-9120-ca0d49065fcd None admin] RBAC: Authorizing `identity:get_group(group_id=2926a97c5f2c4efb95fd77e4efd8210a)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:49.374449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f81109-94b9-4d14-9120-ca0d49065fcd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:49.376042 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 215/452] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:49 2022] GET /identity/v3/groups/2926a97c5f2c4efb95fd77e4efd8210a => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:49.381898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f81109-94b9-4d14-9120-ca0d49065fcd None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:49.435660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:49.457155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343858a-68be-40e1-aabe-814ef7f45548 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:49.495108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a3987-676f-4adc-9ad9-85f6548f4279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:49.495734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a3987-676f-4adc-9ad9-85f6548f4279 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:49.496037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a3987-676f-4adc-9ad9-85f6548f4279 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:49.496306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a3987-676f-4adc-9ad9-85f6548f4279 None admin] PATH_INFO: `/v3/groups/1198d79329074d26a5991f4153c0913c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:49.496393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f81109-94b9-4d14-9120-ca0d49065fcd None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:49.497415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a3987-676f-4adc-9ad9-85f6548f4279 None admin] RBAC: Authorizing `identity:get_group(group_id=1198d79329074d26a5991f4153c0913c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:49.498909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a3987-676f-4adc-9ad9-85f6548f4279 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:49.500467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 238/453] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Aug 8 16:40:49 2022] GET /identity/v3/groups/1198d79329074d26a5991f4153c0913c => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:49.506176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a3987-676f-4adc-9ad9-85f6548f4279 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:49.531638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d202d-947e-49e8-ba93-dc16ffa636f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:49.532235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d202d-947e-49e8-ba93-dc16ffa636f0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:49.532513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d202d-947e-49e8-ba93-dc16ffa636f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:49.532781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d202d-947e-49e8-ba93-dc16ffa636f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:49.533509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d202d-947e-49e8-ba93-dc16ffa636f0 None admin] RBAC: Adding query filter params (group.id=2926a97c5f2c4efb95fd77e4efd8210a, role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=c0b6d65db9ee4a2cb85f51ab2f1fbc92) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:49.533852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d202d-947e-49e8-ba93-dc16ffa636f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:49.535278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d202d-947e-49e8-ba93-dc16ffa636f0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:49.540951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 216/454] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:49 2022] GET /identity/v3/role_assignments?group.id=2926a97c5f2c4efb95fd77e4efd8210a&scope.project.id=c0b6d65db9ee4a2cb85f51ab2f1fbc92&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:49.548947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:49.609894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a3987-676f-4adc-9ad9-85f6548f4279 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:49.644662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcb6e8-85f0-4df2-94d2-cf7baf6d27f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:49.645277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb6e8-85f0-4df2-94d2-cf7baf6d27f1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:49.645553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb6e8-85f0-4df2-94d2-cf7baf6d27f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:49.645843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb6e8-85f0-4df2-94d2-cf7baf6d27f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:49.646575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcb6e8-85f0-4df2-94d2-cf7baf6d27f1 None admin] RBAC: Adding query filter params (group.id=1198d79329074d26a5991f4153c0913c, role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=801abb9eafc94aef9a541cb855fb81cd) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:49.646880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcb6e8-85f0-4df2-94d2-cf7baf6d27f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:49.648335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcb6e8-85f0-4df2-94d2-cf7baf6d27f1 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:49.655386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 239/455] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Aug 8 16:40:49 2022] GET /identity/v3/role_assignments?group.id=1198d79329074d26a5991f4153c0913c&scope.project.id=801abb9eafc94aef9a541cb855fb81cd&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:52.436254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bacd09-f224-482c-b997-971a0651b70b None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:52.436798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bacd09-f224-482c-b997-971a0651b70b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:52.436798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bacd09-f224-482c-b997-971a0651b70b None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:52.438414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 217/456] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:52.443459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f339d-7446-41c5-bc82-cd052011dc9b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:52.443846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f339d-7446-41c5-bc82-cd052011dc9b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:52.444024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f339d-7446-41c5-bc82-cd052011dc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:52.468787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-989f339d-7446-41c5-bc82-cd052011dc9b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:52.470019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-989f339d-7446-41c5-bc82-cd052011dc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:52.518693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592d7e-8126-4a36-8168-977210df26d0 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:52.519058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592d7e-8126-4a36-8168-977210df26d0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:52.519332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592d7e-8126-4a36-8168-977210df26d0 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:52.520929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 218/457] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:52.526117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92fdeb-847e-4104-a29c-2a848bb93a16 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:52.526402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92fdeb-847e-4104-a29c-2a848bb93a16 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:52.526730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92fdeb-847e-4104-a29c-2a848bb93a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:52.546455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 240/458] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:52.558934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5d92fdeb-847e-4104-a29c-2a848bb93a16 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:52.560123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5d92fdeb-847e-4104-a29c-2a848bb93a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:52.573147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f339d-7446-41c5-bc82-cd052011dc9b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:52.622268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:52.648603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 219/459] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:52.650068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5f454-f1bf-4245-a4eb-4bae2554b3b3 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:52.650381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5f454-f1bf-4245-a4eb-4bae2554b3b3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:52.650653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5f454-f1bf-4245-a4eb-4bae2554b3b3 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:52.652750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 220/460] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:52.663264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233179c4-39eb-427a-89d2-be2a0b0e89ef None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:52.663449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233179c4-39eb-427a-89d2-be2a0b0e89ef None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:52.663635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233179c4-39eb-427a-89d2-be2a0b0e89ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:52.688816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-233179c4-39eb-427a-89d2-be2a0b0e89ef None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:52.690044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-233179c4-39eb-427a-89d2-be2a0b0e89ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:52.696411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f339d-7446-41c5-bc82-cd052011dc9b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:52.719340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:52.733619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a258c0b-0086-4b87-a942-f9ab2b76274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:52.734683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a258c0b-0086-4b87-a942-f9ab2b76274a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:52.735161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a258c0b-0086-4b87-a942-f9ab2b76274a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:52.735627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a258c0b-0086-4b87-a942-f9ab2b76274a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:52.746641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a258c0b-0086-4b87-a942-f9ab2b76274a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:52.748446 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a258c0b-0086-4b87-a942-f9ab2b76274a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-6a258c0b-0086-4b87-a942-f9ab2b76274a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:52.764657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:52.765596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:52.766455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:52.766455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:52.766455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:52.768430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 241/462] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:52.768529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 221/462] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:52.773218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233179c4-39eb-427a-89d2-be2a0b0e89ef None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:52.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a258c0b-0086-4b87-a942-f9ab2b76274a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:52.881310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:52.882504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233179c4-39eb-427a-89d2-be2a0b0e89ef None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:52.894104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a258c0b-0086-4b87-a942-f9ab2b76274a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:52.920631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc109e-7075-4ad3-a6ae-dad5da61f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:52.921269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc109e-7075-4ad3-a6ae-dad5da61f8e3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:52.921548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc109e-7075-4ad3-a6ae-dad5da61f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:52.921837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc109e-7075-4ad3-a6ae-dad5da61f8e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:52.932079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2309f8-2493-4f85-acd5-cd316e876557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:52.932945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2309f8-2493-4f85-acd5-cd316e876557 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:52.933441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc109e-7075-4ad3-a6ae-dad5da61f8e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:52.933549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2309f8-2493-4f85-acd5-cd316e876557 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:52.934044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2309f8-2493-4f85-acd5-cd316e876557 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:52.934919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc109e-7075-4ad3-a6ae-dad5da61f8e3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:52.935146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2309f8-2493-4f85-acd5-cd316e876557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:52.935619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2309f8-2493-4f85-acd5-cd316e876557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:52.937153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2309f8-2493-4f85-acd5-cd316e876557 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-00fc109e-7075-4ad3-a6ae-dad5da61f8e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:52.954426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:52.955334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:52.956158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:52.956158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:52.956158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:52.956938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 242/464] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:52.957130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 222/464] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:52.958024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc109e-7075-4ad3-a6ae-dad5da61f8e3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:52.958434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2309f8-2493-4f85-acd5-cd316e876557 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:52.979504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:53.065186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2309f8-2493-4f85-acd5-cd316e876557 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:53.074469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc109e-7075-4ad3-a6ae-dad5da61f8e3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:53.101613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353c7be-a4a6-462d-84d2-2dfa416c179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:53.102237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353c7be-a4a6-462d-84d2-2dfa416c179a None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.102515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353c7be-a4a6-462d-84d2-2dfa416c179a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.102785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353c7be-a4a6-462d-84d2-2dfa416c179a None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.103974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353c7be-a4a6-462d-84d2-2dfa416c179a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:53.105424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353c7be-a4a6-462d-84d2-2dfa416c179a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:53.109335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2d3da-3461-4480-9e10-e418b7b411f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:53.109991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2d3da-3461-4480-9e10-e418b7b411f7 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.110270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2d3da-3461-4480-9e10-e418b7b411f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.110555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2d3da-3461-4480-9e10-e418b7b411f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.121980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2d3da-3461-4480-9e10-e418b7b411f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:53.123430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2d3da-3461-4480-9e10-e418b7b411f7 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b8b2d3da-3461-4480-9e10-e418b7b411f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:53.137261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:53.138808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:53.139631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:53.139631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:53.139631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:53.139631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 223/465] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:53.139631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2d3da-3461-4480-9e10-e418b7b411f7 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:53.197031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 243/466] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:52 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:53.199467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9f6e7-81c4-442d-aa0b-ef5d835f0413 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.200020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9f6e7-81c4-442d-aa0b-ef5d835f0413 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.200422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9f6e7-81c4-442d-aa0b-ef5d835f0413 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.202801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 244/467] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:53.203894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9f6e7-81c4-442d-aa0b-ef5d835f0413 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:53.211305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:53.211900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:53.259377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2d3da-3461-4480-9e10-e418b7b411f7 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:53.302997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6c780-ac91-47cd-b4ac-4937fcb354f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:53.304012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6c780-ac91-47cd-b4ac-4937fcb354f1 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.304544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6c780-ac91-47cd-b4ac-4937fcb354f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.305011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6c780-ac91-47cd-b4ac-4937fcb354f1 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.305937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6c780-ac91-47cd-b4ac-4937fcb354f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:53.307138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6c780-ac91-47cd-b4ac-4937fcb354f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:53.308873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6c780-ac91-47cd-b4ac-4937fcb354f1 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:53.324351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 224/468] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:53.326083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506aea0-cf2b-4372-b6a2-24e2eafebb20 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.327544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506aea0-cf2b-4372-b6a2-24e2eafebb20 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.327544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506aea0-cf2b-4372-b6a2-24e2eafebb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.345991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9f6e7-81c4-442d-aa0b-ef5d835f0413 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:53.347922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1506aea0-cf2b-4372-b6a2-24e2eafebb20 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:53.349458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1506aea0-cf2b-4372-b6a2-24e2eafebb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:53.392230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341d926-836c-4ffd-8329-f159568b926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:53.392443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341d926-836c-4ffd-8329-f159568b926c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.393272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341d926-836c-4ffd-8329-f159568b926c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.393272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341d926-836c-4ffd-8329-f159568b926c None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.393547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341d926-836c-4ffd-8329-f159568b926c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:53.393865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341d926-836c-4ffd-8329-f159568b926c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:53.395255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341d926-836c-4ffd-8329-f159568b926c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:53.408389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 245/469] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:53.409195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341d926-836c-4ffd-8329-f159568b926c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:53.432989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 225/470] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:53.434687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af947f70-8b37-4a24-8404-195e49ffb730 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.434970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af947f70-8b37-4a24-8404-195e49ffb730 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.435240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af947f70-8b37-4a24-8404-195e49ffb730 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.436509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 226/471] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:53.437190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af947f70-8b37-4a24-8404-195e49ffb730 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:53.491235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:53.502431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:53.531486 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341d926-836c-4ffd-8329-f159568b926c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:53.545800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af947f70-8b37-4a24-8404-195e49ffb730 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:53.567831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368dda5-7a7c-42b0-9a13-f7eb42bd92a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:53.568437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368dda5-7a7c-42b0-9a13-f7eb42bd92a5 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.568718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368dda5-7a7c-42b0-9a13-f7eb42bd92a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.569061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368dda5-7a7c-42b0-9a13-f7eb42bd92a5 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.569686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368dda5-7a7c-42b0-9a13-f7eb42bd92a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:53.571154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368dda5-7a7c-42b0-9a13-f7eb42bd92a5 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:53.582844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d2532-39cb-439a-9488-7cf8fada743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:53.583448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d2532-39cb-439a-9488-7cf8fada743d None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.583724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d2532-39cb-439a-9488-7cf8fada743d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.584840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d2532-39cb-439a-9488-7cf8fada743d None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.585516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d2532-39cb-439a-9488-7cf8fada743d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:53.587014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d2532-39cb-439a-9488-7cf8fada743d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:53.676375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 246/472] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:53 2022] POST /identity/v3/users => generated 248 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:53.677696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e0a39-fa76-4807-a9d2-a42757150db6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.678023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e0a39-fa76-4807-a9d2-a42757150db6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.678291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e0a39-fa76-4807-a9d2-a42757150db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.700178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b15e0a39-fa76-4807-a9d2-a42757150db6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:53.701499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b15e0a39-fa76-4807-a9d2-a42757150db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:53.722121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 227/473] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:53 2022] POST /identity/v3/users => generated 246 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:53.722121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d2532-39cb-439a-9488-7cf8fada743d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:53.856956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 247/474] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:53.888282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e0a39-fa76-4807-a9d2-a42757150db6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:53.913245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d2532-39cb-439a-9488-7cf8fada743d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:53.932497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:53.945927 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:53.960606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603f90b-eb6f-44dc-9d3a-25769c3130cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:53.961147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603f90b-eb6f-44dc-9d3a-25769c3130cc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:53.961442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603f90b-eb6f-44dc-9d3a-25769c3130cc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:53.961689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603f90b-eb6f-44dc-9d3a-25769c3130cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:53.974768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603f90b-eb6f-44dc-9d3a-25769c3130cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:53.976509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603f90b-eb6f-44dc-9d3a-25769c3130cc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-5603f90b-eb6f-44dc-9d3a-25769c3130cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:53.998371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:53.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:54.000133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:54.000133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:54.000133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:54.002222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 228/475] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:54.008481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603f90b-eb6f-44dc-9d3a-25769c3130cc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:54.014609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e0a39-fa76-4807-a9d2-a42757150db6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:54.060844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:54.061546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:54.061856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:54.062135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:54.062871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:54.064325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:54.107133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.sql.core [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 16:40:54.107133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 16:40:54.107133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [parameters: {'id': '302f2f1ee7b94dec88ba3bb4f412a232', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 16:40:54.107133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92260) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 16:40:54.135267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603f90b-eb6f-44dc-9d3a-25769c3130cc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application result.read() Aug 08 16:40:54.153471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:40:54.154393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application yield resource Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise self.value Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application session.commit() Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 16:40:54.156908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self.session.flush() Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application compat.raise_( Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 16:40:54.158035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:40:54.158900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application result.read() Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:40:54.159698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [parameters: {'id': '302f2f1ee7b94dec88ba3bb4f412a232', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:54.160444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 16:40:54.162180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:54.164812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 248/476] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Aug 8 16:40:53 2022] POST /identity/v3/roles => generated 144 bytes in 277 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:54.171360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:54.189836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a52349-c981-4d1c-995a-2b21b621423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:54.189836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a52349-c981-4d1c-995a-2b21b621423c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:54.189836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a52349-c981-4d1c-995a-2b21b621423c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:54.189836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a52349-c981-4d1c-995a-2b21b621423c None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:54.190222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a52349-c981-4d1c-995a-2b21b621423c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:54.190222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a52349-c981-4d1c-995a-2b21b621423c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:54.190222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a52349-c981-4d1c-995a-2b21b621423c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:54.203309 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 229/477] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:40:54.208755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a52349-c981-4d1c-995a-2b21b621423c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:54.264467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:54.363519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d028e-289e-45ca-b42e-83ffb927c9bd None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:54.392793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:54.421144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a52349-c981-4d1c-995a-2b21b621423c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:54.435203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9dbe9-18d1-42f2-984f-2ed3adf4dcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:54.435871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9dbe9-18d1-42f2-984f-2ed3adf4dcb0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:54.436153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9dbe9-18d1-42f2-984f-2ed3adf4dcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:54.436431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9dbe9-18d1-42f2-984f-2ed3adf4dcb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:54.460465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9dbe9-18d1-42f2-984f-2ed3adf4dcb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-7ea9dbe9-18d1-42f2-984f-2ed3adf4dcb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:54.465176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:54.466084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:54.466863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 249/478] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:40:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:40:54.473519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9dbe9-18d1-42f2-984f-2ed3adf4dcb0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:54.514603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de3a17-15f1-4819-9e79-5804a3264def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:54.515260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de3a17-15f1-4819-9e79-5804a3264def None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:54.515517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de3a17-15f1-4819-9e79-5804a3264def None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:54.515786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de3a17-15f1-4819-9e79-5804a3264def None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:54.516443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de3a17-15f1-4819-9e79-5804a3264def None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:54.517926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de3a17-15f1-4819-9e79-5804a3264def None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:54.591525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9dbe9-18d1-42f2-984f-2ed3adf4dcb0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:54.630835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4621e5e-f9d4-4e2d-b572-7cfd526241a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:54.632079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4621e5e-f9d4-4e2d-b572-7cfd526241a5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:54.632585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4621e5e-f9d4-4e2d-b572-7cfd526241a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:54.633046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4621e5e-f9d4-4e2d-b572-7cfd526241a5 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:54.633949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4621e5e-f9d4-4e2d-b572-7cfd526241a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:54.635408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4621e5e-f9d4-4e2d-b572-7cfd526241a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:54.637049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4621e5e-f9d4-4e2d-b572-7cfd526241a5 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:54.647570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 250/479] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:54.653051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 230/480] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:54 2022] POST /identity/v3/users => generated 249 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:40:55.644233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bf1f2-508c-4681-82b8-add6de149849 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:55.645692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bf1f2-508c-4681-82b8-add6de149849 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:55.646183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bf1f2-508c-4681-82b8-add6de149849 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:55.647976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 251/481] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:55.653531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01171dd0-95a4-404c-bff6-1e8d643b2ace None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:55.653962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01171dd0-95a4-404c-bff6-1e8d643b2ace None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:55.654169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01171dd0-95a4-404c-bff6-1e8d643b2ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:55.678769 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-01171dd0-95a4-404c-bff6-1e8d643b2ace None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:55.679964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-01171dd0-95a4-404c-bff6-1e8d643b2ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:55.778494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 231/482] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:55.813426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bf1f2-508c-4681-82b8-add6de149849 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:55.857471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:55.938036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bf1f2-508c-4681-82b8-add6de149849 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:55.999857 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa81dd-4516-4ca6-b36e-8a4458d8cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:56.000495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa81dd-4516-4ca6-b36e-8a4458d8cd9a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.000776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa81dd-4516-4ca6-b36e-8a4458d8cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.001047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa81dd-4516-4ca6-b36e-8a4458d8cd9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.022668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa81dd-4516-4ca6-b36e-8a4458d8cd9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-e2fa81dd-4516-4ca6-b36e-8a4458d8cd9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:56.027417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:56.028277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:56.029072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 252/483] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:40:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:56.035092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01171dd0-95a4-404c-bff6-1e8d643b2ace None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:56.064655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:56.200654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715f051-8883-4d29-b1b2-9d38860cff15 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.200863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715f051-8883-4d29-b1b2-9d38860cff15 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.201132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715f051-8883-4d29-b1b2-9d38860cff15 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.202743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 253/484] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:56.207145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01171dd0-95a4-404c-bff6-1e8d643b2ace None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:56.209724 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ac266-62e4-416e-8d51-5f3d8543ef54 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.210031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ac266-62e4-416e-8d51-5f3d8543ef54 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.210304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ac266-62e4-416e-8d51-5f3d8543ef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.246761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d04ac266-62e4-416e-8d51-5f3d8543ef54 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:56.248784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d04ac266-62e4-416e-8d51-5f3d8543ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:56.255439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86166c-d56e-475a-9409-6ec790907f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:56.256267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86166c-d56e-475a-9409-6ec790907f4a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.256727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86166c-d56e-475a-9409-6ec790907f4a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.257161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86166c-d56e-475a-9409-6ec790907f4a None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.258024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86166c-d56e-475a-9409-6ec790907f4a None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:56.258489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86166c-d56e-475a-9409-6ec790907f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:56.260027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86166c-d56e-475a-9409-6ec790907f4a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:56.278160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 232/485] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:40:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:56.283483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86166c-d56e-475a-9409-6ec790907f4a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:56.378061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 254/486] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:40:56.379785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d99cd-3c15-4183-887e-b8380f060e76 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.379785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d99cd-3c15-4183-887e-b8380f060e76 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.380196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d99cd-3c15-4183-887e-b8380f060e76 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.381806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 255/487] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:56.391575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819396-2b3c-45d6-8bc1-1bdac015f622 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.391871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819396-2b3c-45d6-8bc1-1bdac015f622 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.392211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad819396-2b3c-45d6-8bc1-1bdac015f622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.401936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:56.421181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ad819396-2b3c-45d6-8bc1-1bdac015f622 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:56.423201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ad819396-2b3c-45d6-8bc1-1bdac015f622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:56.443779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86166c-d56e-475a-9409-6ec790907f4a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:56.452390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:56.490130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bab75-3e98-4785-b625-c34e063445ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:56.490784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bab75-3e98-4785-b625-c34e063445ca None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.491334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bab75-3e98-4785-b625-c34e063445ca None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.491614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bab75-3e98-4785-b625-c34e063445ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.504467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bab75-3e98-4785-b625-c34e063445ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:56.506194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bab75-3e98-4785-b625-c34e063445ca None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-2f6bab75-3e98-4785-b625-c34e063445ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:56.521254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:56.522191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:56.523006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:56.523006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:56.523006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:56.523171 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 256/488] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:56.523171 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad819396-2b3c-45d6-8bc1-1bdac015f622 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:56.525171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 233/489] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:56.530218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bab75-3e98-4785-b625-c34e063445ca None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:56.640628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bab75-3e98-4785-b625-c34e063445ca None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:56.649467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad819396-2b3c-45d6-8bc1-1bdac015f622 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:56.668670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:56.679095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369f125-37c2-4bc8-8dd3-b74a9821637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:56.679939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369f125-37c2-4bc8-8dd3-b74a9821637c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.680479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369f125-37c2-4bc8-8dd3-b74a9821637c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.681439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369f125-37c2-4bc8-8dd3-b74a9821637c None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.682391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369f125-37c2-4bc8-8dd3-b74a9821637c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:56.682930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369f125-37c2-4bc8-8dd3-b74a9821637c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:56.684758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369f125-37c2-4bc8-8dd3-b74a9821637c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:56.686716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaaa3e3-c10b-434d-b4fd-e49823cfb004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:56.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaaa3e3-c10b-434d-b4fd-e49823cfb004 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.687637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaaa3e3-c10b-434d-b4fd-e49823cfb004 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.688189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaaa3e3-c10b-434d-b4fd-e49823cfb004 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.699638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 234/490] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:56.699932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaaa3e3-c10b-434d-b4fd-e49823cfb004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:56.701030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369f125-37c2-4bc8-8dd3-b74a9821637c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-bbaaa3e3-c10b-434d-b4fd-e49823cfb004 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:56.705191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:56.706073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 257/491] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:40:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:56.711632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaaa3e3-c10b-434d-b4fd-e49823cfb004 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:56.727085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:56.816784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaaa3e3-c10b-434d-b4fd-e49823cfb004 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:56.824005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369f125-37c2-4bc8-8dd3-b74a9821637c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:56.854668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55680f97-e175-4484-a299-eb08c90c09c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:56.855430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55680f97-e175-4484-a299-eb08c90c09c3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.855712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55680f97-e175-4484-a299-eb08c90c09c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.856076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55680f97-e175-4484-a299-eb08c90c09c3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.860985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721cc28-55a8-4ee8-a6b1-403fb43abbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:56.861622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721cc28-55a8-4ee8-a6b1-403fb43abbce None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.861927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721cc28-55a8-4ee8-a6b1-403fb43abbce None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.862196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721cc28-55a8-4ee8-a6b1-403fb43abbce None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.864161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55680f97-e175-4484-a299-eb08c90c09c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:56.865756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55680f97-e175-4484-a299-eb08c90c09c3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:56.873243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721cc28-55a8-4ee8-a6b1-403fb43abbce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b721cc28-55a8-4ee8-a6b1-403fb43abbce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:56.877955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:56.878861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:56.880027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 235/492] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:40:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:56.881051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721cc28-55a8-4ee8-a6b1-403fb43abbce None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-55680f97-e175-4484-a299-eb08c90c09c3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:40:56.882471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:56.883325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 258/493] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6773c46-5d15-4173-aabb-90e86a5947c4 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:56.885905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6773c46-5d15-4173-aabb-90e86a5947c4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:56.889629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6773c46-5d15-4173-aabb-90e86a5947c4 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:56.889629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 259/494] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:56.889629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6773c46-5d15-4173-aabb-90e86a5947c4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:56.930195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:56.975828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:56.985777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721cc28-55a8-4ee8-a6b1-403fb43abbce None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:56.999783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6773c46-5d15-4173-aabb-90e86a5947c4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.023106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351ca11-1577-4c56-bca5-ba541c522c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.023713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351ca11-1577-4c56-bca5-ba541c522c58 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.023991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351ca11-1577-4c56-bca5-ba541c522c58 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.024260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351ca11-1577-4c56-bca5-ba541c522c58 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.025472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351ca11-1577-4c56-bca5-ba541c522c58 None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:57.025850 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351ca11-1577-4c56-bca5-ba541c522c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.027275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351ca11-1577-4c56-bca5-ba541c522c58 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.035771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 236/495] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:40:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:57.036635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351ca11-1577-4c56-bca5-ba541c522c58 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.041848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7f5a0-6bc2-4ad7-91c8-74165813a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.041848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7f5a0-6bc2-4ad7-91c8-74165813a75e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.041848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7f5a0-6bc2-4ad7-91c8-74165813a75e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.041848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7f5a0-6bc2-4ad7-91c8-74165813a75e None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.042523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7f5a0-6bc2-4ad7-91c8-74165813a75e None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:57.042523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7f5a0-6bc2-4ad7-91c8-74165813a75e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.042943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7f5a0-6bc2-4ad7-91c8-74165813a75e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.054260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 260/496] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:40:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:57.055238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d553f-59c2-4693-8f6f-5b3591e09b91 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.055524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d553f-59c2-4693-8f6f-5b3591e09b91 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.055797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d553f-59c2-4693-8f6f-5b3591e09b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.074030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dc5d553f-59c2-4693-8f6f-5b3591e09b91 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:57.075228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dc5d553f-59c2-4693-8f6f-5b3591e09b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:57.143467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 261/497] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:57.147752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351ca11-1577-4c56-bca5-ba541c522c58 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.151533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d553f-59c2-4693-8f6f-5b3591e09b91 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.157320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:57.185243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7015f35-7dd5-417f-8067-283fe1452448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.185925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7015f35-7dd5-417f-8067-283fe1452448 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.186209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7015f35-7dd5-417f-8067-283fe1452448 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.186484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7015f35-7dd5-417f-8067-283fe1452448 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.187242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7015f35-7dd5-417f-8067-283fe1452448 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:57.187550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7015f35-7dd5-417f-8067-283fe1452448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.188976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7015f35-7dd5-417f-8067-283fe1452448 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.226366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 237/498] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:57.227125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7015f35-7dd5-417f-8067-283fe1452448 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.231697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:57.265776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d553f-59c2-4693-8f6f-5b3591e09b91 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.301623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839878ee-6a29-4a68-a9ea-4a0e171e5b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.302298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839878ee-6a29-4a68-a9ea-4a0e171e5b74 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.302547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839878ee-6a29-4a68-a9ea-4a0e171e5b74 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.302813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839878ee-6a29-4a68-a9ea-4a0e171e5b74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.314203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839878ee-6a29-4a68-a9ea-4a0e171e5b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.315701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839878ee-6a29-4a68-a9ea-4a0e171e5b74 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-839878ee-6a29-4a68-a9ea-4a0e171e5b74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:57.331608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:57.332622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:57.333433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:57.333433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:57.333433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:57.337214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 262/499] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:57.337505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7015f35-7dd5-417f-8067-283fe1452448 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.338095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839878ee-6a29-4a68-a9ea-4a0e171e5b74 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.371227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b9b64-d01d-4474-8c65-6525942c6189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.371855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b9b64-d01d-4474-8c65-6525942c6189 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.372726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b9b64-d01d-4474-8c65-6525942c6189 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.372726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b9b64-d01d-4474-8c65-6525942c6189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.382026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b9b64-d01d-4474-8c65-6525942c6189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.383480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b9b64-d01d-4474-8c65-6525942c6189 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-379b9b64-d01d-4474-8c65-6525942c6189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:57.398482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:57.399527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:57.400789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:57.400789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:57.400789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:57.400789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 238/500] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:57.400789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b9b64-d01d-4474-8c65-6525942c6189 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.420213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:57.440755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839878ee-6a29-4a68-a9ea-4a0e171e5b74 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.461343 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:57.475457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854d959-0526-4128-8af1-2a163f998bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.476068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854d959-0526-4128-8af1-2a163f998bb9 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.476345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854d959-0526-4128-8af1-2a163f998bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.476662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854d959-0526-4128-8af1-2a163f998bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.487579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854d959-0526-4128-8af1-2a163f998bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.489019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854d959-0526-4128-8af1-2a163f998bb9 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.501412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b9b64-d01d-4474-8c65-6525942c6189 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-6854d959-0526-4128-8af1-2a163f998bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:57.503881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:57.504903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:57.505736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:57.505736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:57.505736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:57.505736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 263/501] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:57.506393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191f7e2-1f9f-4119-b819-2447328537f2 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.506488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191f7e2-1f9f-4119-b819-2447328537f2 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.506727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191f7e2-1f9f-4119-b819-2447328537f2 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.509837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 264/502] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:40:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:40:57.509837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191f7e2-1f9f-4119-b819-2447328537f2 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.536682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcbd80-0a4f-42b0-850d-a7290decdd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.537291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcbd80-0a4f-42b0-850d-a7290decdd63 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.537575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcbd80-0a4f-42b0-850d-a7290decdd63 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.537881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcbd80-0a4f-42b0-850d-a7290decdd63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.608306 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcbd80-0a4f-42b0-850d-a7290decdd63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.609750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcbd80-0a4f-42b0-850d-a7290decdd63 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.611268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191f7e2-1f9f-4119-b819-2447328537f2 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.628161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:57.644531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a8d9-1e26-42a5-9e65-c6fb1843ae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.645147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a8d9-1e26-42a5-9e65-c6fb1843ae0b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.645426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a8d9-1e26-42a5-9e65-c6fb1843ae0b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.645700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a8d9-1e26-42a5-9e65-c6fb1843ae0b None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.646394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a8d9-1e26-42a5-9e65-c6fb1843ae0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:57.646689 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a8d9-1e26-42a5-9e65-c6fb1843ae0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.648046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a8d9-1e26-42a5-9e65-c6fb1843ae0b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.659353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 265/503] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:57.660033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a8d9-1e26-42a5-9e65-c6fb1843ae0b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.691207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:57.711492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 239/504] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:57.712357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcbd80-0a4f-42b0-850d-a7290decdd63 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.758069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a8d9-1e26-42a5-9e65-c6fb1843ae0b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.791394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7290c-729b-445b-8c36-5b1d05ae34eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.791994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7290c-729b-445b-8c36-5b1d05ae34eb None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.792273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7290c-729b-445b-8c36-5b1d05ae34eb None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.792542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7290c-729b-445b-8c36-5b1d05ae34eb None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.793179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7290c-729b-445b-8c36-5b1d05ae34eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:57.793482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7290c-729b-445b-8c36-5b1d05ae34eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.794894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7290c-729b-445b-8c36-5b1d05ae34eb None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.805734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 266/505] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:57.807549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe8771-84ad-4682-9a55-0de4363aa140 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.807835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe8771-84ad-4682-9a55-0de4363aa140 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.808110 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe8771-84ad-4682-9a55-0de4363aa140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.816345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcbd80-0a4f-42b0-850d-a7290decdd63 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.825180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-e0fe8771-84ad-4682-9a55-0de4363aa140 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:40:57.826380 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e0fe8771-84ad-4682-9a55-0de4363aa140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:40:57.833409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:57.849515 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36e9be-c7f1-4e1e-842f-8a079f587631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:57.850142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36e9be-c7f1-4e1e-842f-8a079f587631 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:57.850420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36e9be-c7f1-4e1e-842f-8a079f587631 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:57.850687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36e9be-c7f1-4e1e-842f-8a079f587631 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:57.851320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36e9be-c7f1-4e1e-842f-8a079f587631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:57.851615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36e9be-c7f1-4e1e-842f-8a079f587631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:57.853555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36e9be-c7f1-4e1e-842f-8a079f587631 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:57.864814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 240/506] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:57.865675 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36e9be-c7f1-4e1e-842f-8a079f587631 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.893493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 267/507] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:40:57.894456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe8771-84ad-4682-9a55-0de4363aa140 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:57.929569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:57.966908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36e9be-c7f1-4e1e-842f-8a079f587631 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:57.999430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968746a-c7e2-4f7f-85f5-57093f388a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.000052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968746a-c7e2-4f7f-85f5-57093f388a48 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.000327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968746a-c7e2-4f7f-85f5-57093f388a48 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.000592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968746a-c7e2-4f7f-85f5-57093f388a48 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.003664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe8771-84ad-4682-9a55-0de4363aa140 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.008132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968746a-c7e2-4f7f-85f5-57093f388a48 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.009576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968746a-c7e2-4f7f-85f5-57093f388a48 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-9968746a-c7e2-4f7f-85f5-57093f388a48 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:40:58.021640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:58.022564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:58.023428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 241/508] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:58.023987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968746a-c7e2-4f7f-85f5-57093f388a48 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.039798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f21559-7343-4f70-ba62-cd30c22b1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.041076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f21559-7343-4f70-ba62-cd30c22b1c88 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.041354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f21559-7343-4f70-ba62-cd30c22b1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.041618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f21559-7343-4f70-ba62-cd30c22b1c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.051436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f21559-7343-4f70-ba62-cd30c22b1c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.052888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f21559-7343-4f70-ba62-cd30c22b1c88 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.065427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-26f21559-7343-4f70-ba62-cd30c22b1c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:58.066912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:58.067798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:58.068651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:58.068651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:58.068651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:58.068651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 268/509] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:58.068867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f21559-7343-4f70-ba62-cd30c22b1c88 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.129885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968746a-c7e2-4f7f-85f5-57093f388a48 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.142608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:58.162691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4fdf7-fd0d-4de9-833e-4e355162f09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.163313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4fdf7-fd0d-4de9-833e-4e355162f09d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.163593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4fdf7-fd0d-4de9-833e-4e355162f09d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.163868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4fdf7-fd0d-4de9-833e-4e355162f09d None admin] PATH_INFO: `/v3/users/nova` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.167576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f21559-7343-4f70-ba62-cd30c22b1c88 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.172521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4fdf7-fd0d-4de9-833e-4e355162f09d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.174003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4fdf7-fd0d-4de9-833e-4e355162f09d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-c3e4fdf7-fd0d-4de9-833e-4e355162f09d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:40:58.184925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:58.186005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:58.186864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 242/510] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Aug 8 16:40:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:58.187413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4fdf7-fd0d-4de9-833e-4e355162f09d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.201509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ca2d-d92c-46e2-9d0c-9f975d7e0b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.202128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ca2d-d92c-46e2-9d0c-9f975d7e0b7f None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.202401 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ca2d-d92c-46e2-9d0c-9f975d7e0b7f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.202664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ca2d-d92c-46e2-9d0c-9f975d7e0b7f None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.203282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875ca2d-d92c-46e2-9d0c-9f975d7e0b7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.204698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875ca2d-d92c-46e2-9d0c-9f975d7e0b7f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.284160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:58.286828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 269/511] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:40:58 2022] POST /identity/v3/users => generated 247 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:40:58.287781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ca2d-d92c-46e2-9d0c-9f975d7e0b7f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.289751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4fdf7-fd0d-4de9-833e-4e355162f09d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.333808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028e482-242e-4d1e-9540-b59fd781c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.334477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028e482-242e-4d1e-9540-b59fd781c20f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.334755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028e482-242e-4d1e-9540-b59fd781c20f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.335023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028e482-242e-4d1e-9540-b59fd781c20f None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.349396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028e482-242e-4d1e-9540-b59fd781c20f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-d028e482-242e-4d1e-9540-b59fd781c20f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:40:58.356020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:58.356977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:58.357759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 243/512] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:40:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:58.358446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028e482-242e-4d1e-9540-b59fd781c20f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.409464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ca2d-d92c-46e2-9d0c-9f975d7e0b7f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.443112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:58.451526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7edc3-1af3-4233-b78d-8f70f8306061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.452127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7edc3-1af3-4233-b78d-8f70f8306061 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.452398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7edc3-1af3-4233-b78d-8f70f8306061 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.457013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7edc3-1af3-4233-b78d-8f70f8306061 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.457013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7edc3-1af3-4233-b78d-8f70f8306061 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:58.457013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7edc3-1af3-4233-b78d-8f70f8306061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.457013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7edc3-1af3-4233-b78d-8f70f8306061 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.471098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028e482-242e-4d1e-9540-b59fd781c20f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.495563 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 270/513] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Aug 8 16:40:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:58.496591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7edc3-1af3-4233-b78d-8f70f8306061 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.514882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84100db9-6e4c-4096-8ff8-d38ca960fae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.515511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84100db9-6e4c-4096-8ff8-d38ca960fae0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.515789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84100db9-6e4c-4096-8ff8-d38ca960fae0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.516120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84100db9-6e4c-4096-8ff8-d38ca960fae0 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.516729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84100db9-6e4c-4096-8ff8-d38ca960fae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:58.517027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84100db9-6e4c-4096-8ff8-d38ca960fae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.518442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84100db9-6e4c-4096-8ff8-d38ca960fae0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.530541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 244/514] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:58.531159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84100db9-6e4c-4096-8ff8-d38ca960fae0 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.558269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:58.615839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7edc3-1af3-4233-b78d-8f70f8306061 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.652762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846c5e9-02cf-4422-bafb-5788fc37c701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.653368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846c5e9-02cf-4422-bafb-5788fc37c701 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.653644 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846c5e9-02cf-4422-bafb-5788fc37c701 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.653955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846c5e9-02cf-4422-bafb-5788fc37c701 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.654682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846c5e9-02cf-4422-bafb-5788fc37c701 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:58.654983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846c5e9-02cf-4422-bafb-5788fc37c701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.656410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846c5e9-02cf-4422-bafb-5788fc37c701 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.656599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84100db9-6e4c-4096-8ff8-d38ca960fae0 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.673888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 271/515] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Aug 8 16:40:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:58.674933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846c5e9-02cf-4422-bafb-5788fc37c701 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.695700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcfd43-1a64-41e5-bfc8-fcfd008c437e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.696308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcfd43-1a64-41e5-bfc8-fcfd008c437e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.696589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcfd43-1a64-41e5-bfc8-fcfd008c437e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.696859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcfd43-1a64-41e5-bfc8-fcfd008c437e None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.697529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcfd43-1a64-41e5-bfc8-fcfd008c437e None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:58.697860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcfd43-1a64-41e5-bfc8-fcfd008c437e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.699476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcfd43-1a64-41e5-bfc8-fcfd008c437e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.707877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 245/516] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:40:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:58.708671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcfd43-1a64-41e5-bfc8-fcfd008c437e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.727789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:58.783957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846c5e9-02cf-4422-bafb-5788fc37c701 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.795761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:58.815439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcfd43-1a64-41e5-bfc8-fcfd008c437e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:58.822209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d7464-c087-4ec6-ab80-67014194e049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.822820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d7464-c087-4ec6-ab80-67014194e049 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.823094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d7464-c087-4ec6-ab80-67014194e049 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.823364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d7464-c087-4ec6-ab80-67014194e049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.851949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178217bd-ae6d-4d0b-b49c-3b0ecbc01da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:58.852582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178217bd-ae6d-4d0b-b49c-3b0ecbc01da0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:58.853403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178217bd-ae6d-4d0b-b49c-3b0ecbc01da0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:58.853676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178217bd-ae6d-4d0b-b49c-3b0ecbc01da0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:58.860482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178217bd-ae6d-4d0b-b49c-3b0ecbc01da0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.861968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178217bd-ae6d-4d0b-b49c-3b0ecbc01da0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-178217bd-ae6d-4d0b-b49c-3b0ecbc01da0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:40:58.873731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:58.874585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:58.875413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:40:58.875413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:40:58.875413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:40:58.875413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:40:58.875413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:40:58.875413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:40:58.875413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:40:58.875413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:58.875413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 246/517] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:58.875907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178217bd-ae6d-4d0b-b49c-3b0ecbc01da0 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:58.896807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d7464-c087-4ec6-ab80-67014194e049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:58.898356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d7464-c087-4ec6-ab80-67014194e049 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:58.949773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:58.985303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178217bd-ae6d-4d0b-b49c-3b0ecbc01da0 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.007209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 272/518] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:59.008402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d7464-c087-4ec6-ab80-67014194e049 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.023674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec1785-0d20-4de2-8f92-f8d26ffc4503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.024287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec1785-0d20-4de2-8f92-f8d26ffc4503 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.025167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec1785-0d20-4de2-8f92-f8d26ffc4503 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.025365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec1785-0d20-4de2-8f92-f8d26ffc4503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.036225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:59.098852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec1785-0d20-4de2-8f92-f8d26ffc4503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.100231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec1785-0d20-4de2-8f92-f8d26ffc4503 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.115739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d7464-c087-4ec6-ab80-67014194e049 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.153599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698974-c282-43b1-a8af-29af810510e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.154246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698974-c282-43b1-a8af-29af810510e3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.154522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698974-c282-43b1-a8af-29af810510e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.154789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698974-c282-43b1-a8af-29af810510e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.164514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3698974-c282-43b1-a8af-29af810510e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.166082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3698974-c282-43b1-a8af-29af810510e3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.173335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-c3698974-c282-43b1-a8af-29af810510e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:59.181088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:59.182002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:59.182840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:59.182840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:59.182840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:59.182840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 273/519] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:59.183048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698974-c282-43b1-a8af-29af810510e3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.210250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 247/520] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:40:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:40:59.211387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec1785-0d20-4de2-8f92-f8d26ffc4503 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.253917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:59.289971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698974-c282-43b1-a8af-29af810510e3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.316058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec1785-0d20-4de2-8f92-f8d26ffc4503 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.324512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881870-115b-4d91-8514-d44a806bbe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.325116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881870-115b-4d91-8514-d44a806bbe7b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.325396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881870-115b-4d91-8514-d44a806bbe7b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.325665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881870-115b-4d91-8514-d44a806bbe7b None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.326402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61881870-115b-4d91-8514-d44a806bbe7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:59.326706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61881870-115b-4d91-8514-d44a806bbe7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.328146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61881870-115b-4d91-8514-d44a806bbe7b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.350676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 274/521] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:59.350676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881870-115b-4d91-8514-d44a806bbe7b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.356114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c104bc-1f2d-40f9-ad94-c89cc061fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.356744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c104bc-1f2d-40f9-ad94-c89cc061fa48 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.357020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c104bc-1f2d-40f9-ad94-c89cc061fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.357286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c104bc-1f2d-40f9-ad94-c89cc061fa48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.367983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c104bc-1f2d-40f9-ad94-c89cc061fa48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.369453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c104bc-1f2d-40f9-ad94-c89cc061fa48 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-96c104bc-1f2d-40f9-ad94-c89cc061fa48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:59.383790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:59.385301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:59.386340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:59.386340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:59.386340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:59.386340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 248/522] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:59.386340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c104bc-1f2d-40f9-ad94-c89cc061fa48 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.414447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:59.461468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881870-115b-4d91-8514-d44a806bbe7b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.487774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:59.497401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c104bc-1f2d-40f9-ad94-c89cc061fa48 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.498314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2e98c-5369-4dd5-b8f5-82e069901048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.499268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2e98c-5369-4dd5-b8f5-82e069901048 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.499364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2e98c-5369-4dd5-b8f5-82e069901048 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.499632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2e98c-5369-4dd5-b8f5-82e069901048 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.500265 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2e98c-5369-4dd5-b8f5-82e069901048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:59.500566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2e98c-5369-4dd5-b8f5-82e069901048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.501966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2e98c-5369-4dd5-b8f5-82e069901048 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.516325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 275/523] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:59.517173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2e98c-5369-4dd5-b8f5-82e069901048 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.534900 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536201f4-4cf6-454d-88b5-bc08e24f323a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.535824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536201f4-4cf6-454d-88b5-bc08e24f323a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.536279 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536201f4-4cf6-454d-88b5-bc08e24f323a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.536725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536201f4-4cf6-454d-88b5-bc08e24f323a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.547543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536201f4-4cf6-454d-88b5-bc08e24f323a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.549186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536201f4-4cf6-454d-88b5-bc08e24f323a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-536201f4-4cf6-454d-88b5-bc08e24f323a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:40:59.564504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:59.565428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:40:59.566600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:40:59.566600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:40:59.566600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:40:59.568228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 249/524] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:59.569137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536201f4-4cf6-454d-88b5-bc08e24f323a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.639749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2e98c-5369-4dd5-b8f5-82e069901048 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.658393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:59.676366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ddea-ef3d-4822-bd46-03e372e1a825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.678116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ddea-ef3d-4822-bd46-03e372e1a825 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.678556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ddea-ef3d-4822-bd46-03e372e1a825 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.678982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ddea-ef3d-4822-bd46-03e372e1a825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.679876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22ddea-ef3d-4822-bd46-03e372e1a825 None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=73bfe655c6d34918b21acb8748b9156d) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:59.679987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536201f4-4cf6-454d-88b5-bc08e24f323a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.680409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22ddea-ef3d-4822-bd46-03e372e1a825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.683396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22ddea-ef3d-4822-bd46-03e372e1a825 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.697820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 276/525] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/role_assignments?user.id=73bfe655c6d34918b21acb8748b9156d&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:59.697820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ddea-ef3d-4822-bd46-03e372e1a825 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.725991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19a8bf-e14d-4e36-aa3c-ef627da96714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.726721 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19a8bf-e14d-4e36-aa3c-ef627da96714 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.727086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19a8bf-e14d-4e36-aa3c-ef627da96714 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.727511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19a8bf-e14d-4e36-aa3c-ef627da96714 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.728383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19a8bf-e14d-4e36-aa3c-ef627da96714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:59.728750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19a8bf-e14d-4e36-aa3c-ef627da96714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.730308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19a8bf-e14d-4e36-aa3c-ef627da96714 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.744763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 250/526] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:59.745744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19a8bf-e14d-4e36-aa3c-ef627da96714 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.766962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:40:59.837816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ddea-ef3d-4822-bd46-03e372e1a825 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.862157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19a8bf-e14d-4e36-aa3c-ef627da96714 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:40:59.889176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d5a45-6d37-496c-9004-ee363ac0e8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.890648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d5a45-6d37-496c-9004-ee363ac0e8f5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.890648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d5a45-6d37-496c-9004-ee363ac0e8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.890648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d5a45-6d37-496c-9004-ee363ac0e8f5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.898642 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d5a45-6d37-496c-9004-ee363ac0e8f5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.900123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d5a45-6d37-496c-9004-ee363ac0e8f5 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.911431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c23d8-df38-4079-b36e-40fe55e62a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-e50d5a45-6d37-496c-9004-ee363ac0e8f5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:40:59.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:40:59.912537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:40:59.913394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 277/527] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:40:59.915639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c23d8-df38-4079-b36e-40fe55e62a13 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:40:59.916128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c23d8-df38-4079-b36e-40fe55e62a13 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:40:59.916711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c23d8-df38-4079-b36e-40fe55e62a13 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:40:59.917659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c23d8-df38-4079-b36e-40fe55e62a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:40:59.918243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c23d8-df38-4079-b36e-40fe55e62a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:40:59.918656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d5a45-6d37-496c-9004-ee363ac0e8f5 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.920505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c23d8-df38-4079-b36e-40fe55e62a13 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:40:59.937842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 251/528] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:40:59.938993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c23d8-df38-4079-b36e-40fe55e62a13 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:40:59.976048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:00.025103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:00.042363 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d5a45-6d37-496c-9004-ee363ac0e8f5 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:00.074478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c23d8-df38-4079-b36e-40fe55e62a13 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:00.085177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f033-768d-49ad-aeb4-b31111d8e9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:00.085492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f033-768d-49ad-aeb4-b31111d8e9e6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.085767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f033-768d-49ad-aeb4-b31111d8e9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.086135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f033-768d-49ad-aeb4-b31111d8e9e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.093574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f033-768d-49ad-aeb4-b31111d8e9e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:00.095174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f033-768d-49ad-aeb4-b31111d8e9e6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-9fb7f033-768d-49ad-aeb4-b31111d8e9e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:00.107803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:00.108664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:00.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:00.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:00.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:00.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:00.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:00.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:00.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:00.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:00.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 278/529] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:00.110463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f033-768d-49ad-aeb4-b31111d8e9e6 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:00.123841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e83838-56ba-43c6-9cf0-431345cb1b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:00.125441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e83838-56ba-43c6-9cf0-431345cb1b4c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.125940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e83838-56ba-43c6-9cf0-431345cb1b4c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.126382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e83838-56ba-43c6-9cf0-431345cb1b4c None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.127531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e83838-56ba-43c6-9cf0-431345cb1b4c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:00.128498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e83838-56ba-43c6-9cf0-431345cb1b4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:00.130391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e83838-56ba-43c6-9cf0-431345cb1b4c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:00.155158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 252/530] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Aug 8 16:40:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:00.156231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e83838-56ba-43c6-9cf0-431345cb1b4c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:00.226179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f033-768d-49ad-aeb4-b31111d8e9e6 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:00.228958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:00.264630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e96db6-07f0-4199-8a54-610b326dfda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:00.265289 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e96db6-07f0-4199-8a54-610b326dfda6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.266040 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e96db6-07f0-4199-8a54-610b326dfda6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.266608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e96db6-07f0-4199-8a54-610b326dfda6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.274076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e96db6-07f0-4199-8a54-610b326dfda6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:00.276163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e96db6-07f0-4199-8a54-610b326dfda6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:00.287588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e83838-56ba-43c6-9cf0-431345cb1b4c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-83e96db6-07f0-4199-8a54-610b326dfda6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:00.288028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:00.295144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 279/531] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:00.296001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e96db6-07f0-4199-8a54-610b326dfda6 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:00.319981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:00.337479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e910e-e31f-495a-9104-cad5109453ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:00.338472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e910e-e31f-495a-9104-cad5109453ab None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.338745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e910e-e31f-495a-9104-cad5109453ab None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.339013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e910e-e31f-495a-9104-cad5109453ab None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.339705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e910e-e31f-495a-9104-cad5109453ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:00.340002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e910e-e31f-495a-9104-cad5109453ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:00.341426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e910e-e31f-495a-9104-cad5109453ab None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:00.361852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 253/532] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:00.362907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e910e-e31f-495a-9104-cad5109453ab None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:00.411215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e96db6-07f0-4199-8a54-610b326dfda6 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:00.445648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08c85-3bb6-432a-bd7b-fe152c36a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:00.446292 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08c85-3bb6-432a-bd7b-fe152c36a02a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.446572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08c85-3bb6-432a-bd7b-fe152c36a02a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.446941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08c85-3bb6-432a-bd7b-fe152c36a02a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.466099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e910e-e31f-495a-9104-cad5109453ab None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:00.481850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:00.502474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44dc214-fc8f-4e48-b2ba-f72cd3f55ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:00.503184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dc214-fc8f-4e48-b2ba-f72cd3f55ff8 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.503476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dc214-fc8f-4e48-b2ba-f72cd3f55ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.503886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dc214-fc8f-4e48-b2ba-f72cd3f55ff8 None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.504415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44dc214-fc8f-4e48-b2ba-f72cd3f55ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:00.504721 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44dc214-fc8f-4e48-b2ba-f72cd3f55ff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:00.506184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44dc214-fc8f-4e48-b2ba-f72cd3f55ff8 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:00.522786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 254/533] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:00.522786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afa9f5-d4c0-43ee-a6c1-31f0c209c0b8 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.522786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afa9f5-d4c0-43ee-a6c1-31f0c209c0b8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.522786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afa9f5-d4c0-43ee-a6c1-31f0c209c0b8 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.522786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 255/534] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:00.522786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afa9f5-d4c0-43ee-a6c1-31f0c209c0b8 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:00.530205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b08c85-3bb6-432a-bd7b-fe152c36a02a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:00.531657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b08c85-3bb6-432a-bd7b-fe152c36a02a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:00.570781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:00.635845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 280/535] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:41:00.636918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08c85-3bb6-432a-bd7b-fe152c36a02a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:00.645622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afa9f5-d4c0-43ee-a6c1-31f0c209c0b8 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:00.682742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03940d40-305b-4e83-a147-e552fb04e290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:00.683596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03940d40-305b-4e83-a147-e552fb04e290 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.683874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03940d40-305b-4e83-a147-e552fb04e290 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.684155 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03940d40-305b-4e83-a147-e552fb04e290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.685234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03940d40-305b-4e83-a147-e552fb04e290 None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=e860aad24688491c9a89ddfb35922eb4) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:00.685539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03940d40-305b-4e83-a147-e552fb04e290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:00.686986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03940d40-305b-4e83-a147-e552fb04e290 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:00.692797 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 256/536] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:00 2022] GET /identity/v3/role_assignments?user.id=e860aad24688491c9a89ddfb35922eb4&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:00.693733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9d76b-b870-48e9-972e-e77dc0556828 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.694047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9d76b-b870-48e9-972e-e77dc0556828 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.694313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9d76b-b870-48e9-972e-e77dc0556828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.715649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-42a9d76b-b870-48e9-972e-e77dc0556828 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:00.716718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:00.717251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-42a9d76b-b870-48e9-972e-e77dc0556828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:00.787398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08c85-3bb6-432a-bd7b-fe152c36a02a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:00.814218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 257/537] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:00.815217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9d76b-b870-48e9-972e-e77dc0556828 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:00.832253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a5fe4-578e-4e0c-9a53-1c074607c2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:00.833240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a5fe4-578e-4e0c-9a53-1c074607c2ae None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:00.833722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a5fe4-578e-4e0c-9a53-1c074607c2ae None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:00.834335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a5fe4-578e-4e0c-9a53-1c074607c2ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:00.835324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a5fe4-578e-4e0c-9a53-1c074607c2ae None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=54b05cbc41b844d8932ba4c15d960289) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:00.835851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a5fe4-578e-4e0c-9a53-1c074607c2ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:00.838814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a5fe4-578e-4e0c-9a53-1c074607c2ae None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:00.847311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 281/538] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:00 2022] GET /identity/v3/role_assignments?user.id=54b05cbc41b844d8932ba4c15d960289&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:00.849004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a5fe4-578e-4e0c-9a53-1c074607c2ae None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:00.902245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:00.965382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9d76b-b870-48e9-972e-e77dc0556828 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:00.994095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a5fe4-578e-4e0c-9a53-1c074607c2ae None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:01.024820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:01.033745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a13a8-0291-4e31-9ecb-e7a98dc0fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:01.034535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a13a8-0291-4e31-9ecb-e7a98dc0fa5f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:01.034809 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a13a8-0291-4e31-9ecb-e7a98dc0fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:01.035076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a13a8-0291-4e31-9ecb-e7a98dc0fa5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:01.047870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a13a8-0291-4e31-9ecb-e7a98dc0fa5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-649a13a8-0291-4e31-9ecb-e7a98dc0fa5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:01.052759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:01.053637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:01.054941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 282/539] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:01.060711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a13a8-0291-4e31-9ecb-e7a98dc0fa5f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:01.070496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea6996-8ad0-4ab3-b358-cc17a9baffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:01.071148 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea6996-8ad0-4ab3-b358-cc17a9baffbb None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:01.071995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea6996-8ad0-4ab3-b358-cc17a9baffbb None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:01.072272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea6996-8ad0-4ab3-b358-cc17a9baffbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:01.093354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea6996-8ad0-4ab3-b358-cc17a9baffbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:01.094857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea6996-8ad0-4ab3-b358-cc17a9baffbb None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-4fea6996-8ad0-4ab3-b358-cc17a9baffbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:01.119992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:01.120946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:01.121769 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:01.121769 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:01.121769 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:01.125677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 258/540] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:01.138807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea6996-8ad0-4ab3-b358-cc17a9baffbb None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:01.226760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a13a8-0291-4e31-9ecb-e7a98dc0fa5f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:01.285545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f02a8-cbb9-4614-88a2-e2e573d9ca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:01.286172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f02a8-cbb9-4614-88a2-e2e573d9ca8d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:01.286444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f02a8-cbb9-4614-88a2-e2e573d9ca8d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:01.286706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f02a8-cbb9-4614-88a2-e2e573d9ca8d None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:01.287364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f02a8-cbb9-4614-88a2-e2e573d9ca8d None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:01.290349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f02a8-cbb9-4614-88a2-e2e573d9ca8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:01.290349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f02a8-cbb9-4614-88a2-e2e573d9ca8d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:01.294773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:01.306148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 283/541] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:01.311409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f02a8-cbb9-4614-88a2-e2e573d9ca8d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:01.317073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea6996-8ad0-4ab3-b358-cc17a9baffbb None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:01.377712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75390df2-b2e3-46e4-96ab-a0c905671889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:01.378340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75390df2-b2e3-46e4-96ab-a0c905671889 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:01.378611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75390df2-b2e3-46e4-96ab-a0c905671889 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:01.378876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75390df2-b2e3-46e4-96ab-a0c905671889 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:01.379521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75390df2-b2e3-46e4-96ab-a0c905671889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:01.379817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75390df2-b2e3-46e4-96ab-a0c905671889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:01.381171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75390df2-b2e3-46e4-96ab-a0c905671889 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:01.405071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 259/542] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:01.412316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75390df2-b2e3-46e4-96ab-a0c905671889 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:01.412783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:01.501534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f02a8-cbb9-4614-88a2-e2e573d9ca8d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:01.564916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a32e0-a986-41dc-970c-119444917e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:01.565601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a32e0-a986-41dc-970c-119444917e41 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:01.565912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a32e0-a986-41dc-970c-119444917e41 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:01.566180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a32e0-a986-41dc-970c-119444917e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:01.583832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a32e0-a986-41dc-970c-119444917e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:01.585463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a32e0-a986-41dc-970c-119444917e41 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-431a32e0-a986-41dc-970c-119444917e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:01.606295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:01.607227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:01.608259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:01.608259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:01.608259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:01.608259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 284/543] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:01.614474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75390df2-b2e3-46e4-96ab-a0c905671889 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:01.614734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a32e0-a986-41dc-970c-119444917e41 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:01.676376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ab197-4e14-49f1-b6d0-22c6a38253f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:01.677006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ab197-4e14-49f1-b6d0-22c6a38253f0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:01.677277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ab197-4e14-49f1-b6d0-22c6a38253f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:01.677541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ab197-4e14-49f1-b6d0-22c6a38253f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:01.691003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ab197-4e14-49f1-b6d0-22c6a38253f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:01.692447 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ab197-4e14-49f1-b6d0-22c6a38253f0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-fd4ab197-4e14-49f1-b6d0-22c6a38253f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:01.710434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:01.711297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:01.712768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:01.712768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:01.712768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:01.712768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:01.712768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:01.712768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:01.712768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:01.712768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:01.713399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 260/544] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:01.720649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ab197-4e14-49f1-b6d0-22c6a38253f0 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:01.742132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:01.796510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:01.800050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a32e0-a986-41dc-970c-119444917e41 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:01.848236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5980a-b15c-44ff-a9f3-34fee5f656f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:01.848835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5980a-b15c-44ff-a9f3-34fee5f656f9 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:01.849107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5980a-b15c-44ff-a9f3-34fee5f656f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:01.849371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5980a-b15c-44ff-a9f3-34fee5f656f9 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:01.850024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5980a-b15c-44ff-a9f3-34fee5f656f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:01.850317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5980a-b15c-44ff-a9f3-34fee5f656f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:01.851675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5980a-b15c-44ff-a9f3-34fee5f656f9 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:01.873106 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 285/545] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:01.880447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5980a-b15c-44ff-a9f3-34fee5f656f9 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:01.917566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ab197-4e14-49f1-b6d0-22c6a38253f0 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:01.964554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c34bb0-a5bc-4d6f-b508-f1458062084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:01.965482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c34bb0-a5bc-4d6f-b508-f1458062084a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:01.965974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c34bb0-a5bc-4d6f-b508-f1458062084a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:01.966411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c34bb0-a5bc-4d6f-b508-f1458062084a None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:01.967267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c34bb0-a5bc-4d6f-b508-f1458062084a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:01.967790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c34bb0-a5bc-4d6f-b508-f1458062084a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:01.969550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c34bb0-a5bc-4d6f-b508-f1458062084a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:01.985810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 261/546] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:01.985810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6ac63-8505-4766-b8fa-b4aaa16a9a3a None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:01.985810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6ac63-8505-4766-b8fa-b4aaa16a9a3a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:01.985810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6ac63-8505-4766-b8fa-b4aaa16a9a3a None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:01.985810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 262/547] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:01.990116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6ac63-8505-4766-b8fa-b4aaa16a9a3a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:02.034025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5980a-b15c-44ff-a9f3-34fee5f656f9 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:02.065811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:02.083671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09779c-801a-4b66-8c49-6b6010cc4d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:02.084448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09779c-801a-4b66-8c49-6b6010cc4d49 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:02.084729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09779c-801a-4b66-8c49-6b6010cc4d49 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:02.084999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09779c-801a-4b66-8c49-6b6010cc4d49 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:02.096452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09779c-801a-4b66-8c49-6b6010cc4d49 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:02.098789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09779c-801a-4b66-8c49-6b6010cc4d49 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:02.107465 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6ac63-8505-4766-b8fa-b4aaa16a9a3a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-fe09779c-801a-4b66-8c49-6b6010cc4d49 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:02.114107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:02.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:02.116212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:02.123072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 286/548] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:02.124503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e385b-554c-4ae1-bc60-6812dcda7e72 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:02.124926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e385b-554c-4ae1-bc60-6812dcda7e72 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:02.125332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e385b-554c-4ae1-bc60-6812dcda7e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:02.146686 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c74ac8-9366-49c3-bb23-cbae04d95cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:02.146933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-974e385b-554c-4ae1-bc60-6812dcda7e72 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:02.147343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c74ac8-9366-49c3-bb23-cbae04d95cdc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:02.147622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c74ac8-9366-49c3-bb23-cbae04d95cdc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:02.147891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c74ac8-9366-49c3-bb23-cbae04d95cdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:02.148665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c74ac8-9366-49c3-bb23-cbae04d95cdc None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=f50ee57e2418417cbe68915ca3f00f76) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:02.148890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-974e385b-554c-4ae1-bc60-6812dcda7e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:02.149008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c74ac8-9366-49c3-bb23-cbae04d95cdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:02.151089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c74ac8-9366-49c3-bb23-cbae04d95cdc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:02.161812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 263/549] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:01 2022] GET /identity/v3/role_assignments?user.id=f50ee57e2418417cbe68915ca3f00f76&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:02.161812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c74ac8-9366-49c3-bb23-cbae04d95cdc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:02.182755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:02.263391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 287/550] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:02.290874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e385b-554c-4ae1-bc60-6812dcda7e72 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:02.363640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c74ac8-9366-49c3-bb23-cbae04d95cdc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:02.404093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:02.412138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7802a1e-a54d-467b-b1f4-78e0820e0bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:02.412989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7802a1e-a54d-467b-b1f4-78e0820e0bac None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:02.417550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7802a1e-a54d-467b-b1f4-78e0820e0bac None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:02.417550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7802a1e-a54d-467b-b1f4-78e0820e0bac None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:02.417550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7802a1e-a54d-467b-b1f4-78e0820e0bac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:02.417550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7802a1e-a54d-467b-b1f4-78e0820e0bac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:02.417550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7802a1e-a54d-467b-b1f4-78e0820e0bac None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:02.439477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 264/551] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Aug 8 16:41:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:02.440440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7802a1e-a54d-467b-b1f4-78e0820e0bac None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:02.457206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e385b-554c-4ae1-bc60-6812dcda7e72 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:02.523240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8c660-11c1-452a-ba3c-3e24cb983bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:02.525181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8c660-11c1-452a-ba3c-3e24cb983bca None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:02.525453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8c660-11c1-452a-ba3c-3e24cb983bca None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:02.525709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8c660-11c1-452a-ba3c-3e24cb983bca None admin] PATH_INFO: `/v3/roles/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:02.548405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8c660-11c1-452a-ba3c-3e24cb983bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-26c8c660-11c1-452a-ba3c-3e24cb983bca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:02.552992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:02.553915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:02.554747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 288/552] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:02.563472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8c660-11c1-452a-ba3c-3e24cb983bca None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:02.589220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:02.609036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7802a1e-a54d-467b-b1f4-78e0820e0bac None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:02.670634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863bd71-c196-4e96-8205-a8bbee245eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:02.671172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863bd71-c196-4e96-8205-a8bbee245eda None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:02.671449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863bd71-c196-4e96-8205-a8bbee245eda None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:02.671717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863bd71-c196-4e96-8205-a8bbee245eda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:02.740879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8c660-11c1-452a-ba3c-3e24cb983bca None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:02.788823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aedc07-3b70-40f9-87a9-11351af90338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:02.789433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aedc07-3b70-40f9-87a9-11351af90338 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:02.789712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aedc07-3b70-40f9-87a9-11351af90338 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:02.790623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aedc07-3b70-40f9-87a9-11351af90338 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:02.791588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aedc07-3b70-40f9-87a9-11351af90338 None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:02.791889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aedc07-3b70-40f9-87a9-11351af90338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:02.794068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aedc07-3b70-40f9-87a9-11351af90338 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:02.795976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:02.806969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 289/553] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:02.812863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aedc07-3b70-40f9-87a9-11351af90338 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:02.834643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863bd71-c196-4e96-8205-a8bbee245eda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:02.836122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863bd71-c196-4e96-8205-a8bbee245eda None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:02.894374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:02.930534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aedc07-3b70-40f9-87a9-11351af90338 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:02.974021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041557da-7676-439e-9a2f-a12b84b278bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:02.975115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041557da-7676-439e-9a2f-a12b84b278bc None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:02.975394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041557da-7676-439e-9a2f-a12b84b278bc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:02.975939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041557da-7676-439e-9a2f-a12b84b278bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:02.989913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041557da-7676-439e-9a2f-a12b84b278bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:02.992273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041557da-7676-439e-9a2f-a12b84b278bc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-041557da-7676-439e-9a2f-a12b84b278bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:03.011354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:03.012405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:03.013264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:03.013264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:03.013264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:03.017812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 290/554] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:03.018852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a472a4-eeed-4c37-8a74-b6a32bf0cd7a None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.019141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a472a4-eeed-4c37-8a74-b6a32bf0cd7a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.019411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a472a4-eeed-4c37-8a74-b6a32bf0cd7a None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.020807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 291/555] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:03.022345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a472a4-eeed-4c37-8a74-b6a32bf0cd7a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.034021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 265/556] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 16:41:03.035346 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12c9cf-2099-4f57-a946-e510dc9c2692 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.035655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12c9cf-2099-4f57-a946-e510dc9c2692 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.035897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12c9cf-2099-4f57-a946-e510dc9c2692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.059495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-9a12c9cf-2099-4f57-a946-e510dc9c2692 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:03.061267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9a12c9cf-2099-4f57-a946-e510dc9c2692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:03.145294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 266/557] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:03.146454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12c9cf-2099-4f57-a946-e510dc9c2692 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.155598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a472a4-eeed-4c37-8a74-b6a32bf0cd7a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:03.178553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:03.204145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:03.206463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b28430-6d24-401a-88e3-073dcf5c1849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:03.207552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b28430-6d24-401a-88e3-073dcf5c1849 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.208141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b28430-6d24-401a-88e3-073dcf5c1849 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.208751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b28430-6d24-401a-88e3-073dcf5c1849 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.209988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b28430-6d24-401a-88e3-073dcf5c1849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:03.210641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b28430-6d24-401a-88e3-073dcf5c1849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:03.213101 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b28430-6d24-401a-88e3-073dcf5c1849 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:03.229339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 292/558] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:03.230330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b28430-6d24-401a-88e3-073dcf5c1849 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.268192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12c9cf-2099-4f57-a946-e510dc9c2692 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:03.301392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2e5c1-8884-4dcf-ba1c-acb5ea984bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:03.302104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2e5c1-8884-4dcf-ba1c-acb5ea984bef None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.302376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2e5c1-8884-4dcf-ba1c-acb5ea984bef None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.302642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2e5c1-8884-4dcf-ba1c-acb5ea984bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.312933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2e5c1-8884-4dcf-ba1c-acb5ea984bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:03.314401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2e5c1-8884-4dcf-ba1c-acb5ea984bef None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-67b2e5c1-8884-4dcf-ba1c-acb5ea984bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:03.329567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:03.331083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:03.331946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:03.331946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:03.331946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:03.331946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 267/559] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:03.332223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6fd20-6908-4a25-a9c8-4ed0fc974c0a None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.332507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6fd20-6908-4a25-a9c8-4ed0fc974c0a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.332778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6fd20-6908-4a25-a9c8-4ed0fc974c0a None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.337881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 268/560] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:03.337881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6fd20-6908-4a25-a9c8-4ed0fc974c0a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.342542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b28430-6d24-401a-88e3-073dcf5c1849 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:03.378264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04431a0-1251-4913-8539-ea53fcd6fba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:03.379059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04431a0-1251-4913-8539-ea53fcd6fba3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.379340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04431a0-1251-4913-8539-ea53fcd6fba3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.379613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04431a0-1251-4913-8539-ea53fcd6fba3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.393801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04431a0-1251-4913-8539-ea53fcd6fba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-b04431a0-1251-4913-8539-ea53fcd6fba3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:03.400010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:03.400897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:03.401823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 293/561] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:03.402505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04431a0-1251-4913-8539-ea53fcd6fba3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.449278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:03.453390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:03.480659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6fd20-6908-4a25-a9c8-4ed0fc974c0a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:03.510352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04431a0-1251-4913-8539-ea53fcd6fba3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:03.519669 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce9c48-216a-49bf-b497-1825a4331c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:03.520301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce9c48-216a-49bf-b497-1825a4331c28 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.520577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce9c48-216a-49bf-b497-1825a4331c28 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.520842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce9c48-216a-49bf-b497-1825a4331c28 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.530025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce9c48-216a-49bf-b497-1825a4331c28 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:03.531457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce9c48-216a-49bf-b497-1825a4331c28 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-cdce9c48-216a-49bf-b497-1825a4331c28 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:03.543335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:03.544296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 269/562] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e454ed-a47e-4211-99c3-5316b5e56b22 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.550213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e454ed-a47e-4211-99c3-5316b5e56b22 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.550921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e454ed-a47e-4211-99c3-5316b5e56b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.552236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419fa3c-54c0-43f8-8ca5-223a95042b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:03.552842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419fa3c-54c0-43f8-8ca5-223a95042b14 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.553175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419fa3c-54c0-43f8-8ca5-223a95042b14 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.553448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419fa3c-54c0-43f8-8ca5-223a95042b14 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.554121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419fa3c-54c0-43f8-8ca5-223a95042b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:03.554471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419fa3c-54c0-43f8-8ca5-223a95042b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:03.556493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419fa3c-54c0-43f8-8ca5-223a95042b14 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:03.566324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-06e454ed-a47e-4211-99c3-5316b5e56b22 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:03.567500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-06e454ed-a47e-4211-99c3-5316b5e56b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:03.568713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 294/563] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:03.574478 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419fa3c-54c0-43f8-8ca5-223a95042b14 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.641618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 270/564] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:03.642768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e454ed-a47e-4211-99c3-5316b5e56b22 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.688417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419fa3c-54c0-43f8-8ca5-223a95042b14 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:03.690893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:03.719264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:03.728192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac610540-ada0-49f2-9c6c-ca84799f2564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:03.728791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac610540-ada0-49f2-9c6c-ca84799f2564 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.729065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac610540-ada0-49f2-9c6c-ca84799f2564 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.729332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac610540-ada0-49f2-9c6c-ca84799f2564 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.730018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac610540-ada0-49f2-9c6c-ca84799f2564 None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:03.730319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac610540-ada0-49f2-9c6c-ca84799f2564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:03.731831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac610540-ada0-49f2-9c6c-ca84799f2564 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:03.739556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 295/565] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:03.740331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac610540-ada0-49f2-9c6c-ca84799f2564 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.766684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e454ed-a47e-4211-99c3-5316b5e56b22 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:03.803815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a21189-02df-44c3-9725-7f0fab37fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:03.804415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a21189-02df-44c3-9725-7f0fab37fc45 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.804699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a21189-02df-44c3-9725-7f0fab37fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.804954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a21189-02df-44c3-9725-7f0fab37fc45 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.805697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a21189-02df-44c3-9725-7f0fab37fc45 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:03.806038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a21189-02df-44c3-9725-7f0fab37fc45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:03.807448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a21189-02df-44c3-9725-7f0fab37fc45 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:03.829469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 271/566] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:03.830146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a21189-02df-44c3-9725-7f0fab37fc45 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.848773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac610540-ada0-49f2-9c6c-ca84799f2564 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:03.883816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b89da-4846-449a-b6a9-b3efe3e1e5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:03.884443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b89da-4846-449a-b6a9-b3efe3e1e5e9 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.884718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b89da-4846-449a-b6a9-b3efe3e1e5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.884985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b89da-4846-449a-b6a9-b3efe3e1e5e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.891740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b89da-4846-449a-b6a9-b3efe3e1e5e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:03.893174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b89da-4846-449a-b6a9-b3efe3e1e5e9 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-ff0b89da-4846-449a-b6a9-b3efe3e1e5e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:03.904341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:03.905242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:03.906159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:03.906159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:03.906159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:03.906159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:03.906159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:03.906159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:03.906159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:03.906159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:03.906159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 296/567] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:03.906636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b89da-4846-449a-b6a9-b3efe3e1e5e9 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:03.938953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:03.951842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a21189-02df-44c3-9725-7f0fab37fc45 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:03.972501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:03.986980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d381025-46d8-4ad1-bd81-7df239fa5214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:03.987536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d381025-46d8-4ad1-bd81-7df239fa5214 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:03.987813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d381025-46d8-4ad1-bd81-7df239fa5214 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:03.988081 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d381025-46d8-4ad1-bd81-7df239fa5214 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:03.998359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d381025-46d8-4ad1-bd81-7df239fa5214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-2d381025-46d8-4ad1-bd81-7df239fa5214 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:04.003251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:04.004133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 272/568] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:04.010157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d381025-46d8-4ad1-bd81-7df239fa5214 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.024729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b89da-4846-449a-b6a9-b3efe3e1e5e9 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.060184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d9873-5cf8-4b9c-9a9d-9aefb7a2d07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.060895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d9873-5cf8-4b9c-9a9d-9aefb7a2d07f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.061255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d9873-5cf8-4b9c-9a9d-9aefb7a2d07f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.062035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d9873-5cf8-4b9c-9a9d-9aefb7a2d07f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.072555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d9873-5cf8-4b9c-9a9d-9aefb7a2d07f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.074396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d9873-5cf8-4b9c-9a9d-9aefb7a2d07f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-c64d9873-5cf8-4b9c-9a9d-9aefb7a2d07f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:04.090659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:04.091804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:04.092851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:04.092851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:04.092851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:04.098549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 297/569] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:04.098549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d9873-5cf8-4b9c-9a9d-9aefb7a2d07f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.118909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d381025-46d8-4ad1-bd81-7df239fa5214 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.157720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277fbf7-2fea-436c-aed7-57a2dad40e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.158391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277fbf7-2fea-436c-aed7-57a2dad40e91 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.158808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277fbf7-2fea-436c-aed7-57a2dad40e91 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.159088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277fbf7-2fea-436c-aed7-57a2dad40e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.172301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277fbf7-2fea-436c-aed7-57a2dad40e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.174187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277fbf7-2fea-436c-aed7-57a2dad40e91 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.176185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-4277fbf7-2fea-436c-aed7-57a2dad40e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:04.193532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:04.194877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:04.201330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:04.201330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:04.201330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:04.201330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 273/570] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:04.201330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277fbf7-2fea-436c-aed7-57a2dad40e91 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.201660 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d9873-5cf8-4b9c-9a9d-9aefb7a2d07f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.233057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bb952-6d1b-4d5e-902a-75f828b5c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.233747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bb952-6d1b-4d5e-902a-75f828b5c9a0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.234110 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bb952-6d1b-4d5e-902a-75f828b5c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.234439 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bb952-6d1b-4d5e-902a-75f828b5c9a0 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.235567 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bb952-6d1b-4d5e-902a-75f828b5c9a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:04.235731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bb952-6d1b-4d5e-902a-75f828b5c9a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.237257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bb952-6d1b-4d5e-902a-75f828b5c9a0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.238412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:04.250270 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 298/571] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:04.251057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bb952-6d1b-4d5e-902a-75f828b5c9a0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.312693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277fbf7-2fea-436c-aed7-57a2dad40e91 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.350736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3498e-e6f8-43ae-9caa-1d3485a5db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.351343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3498e-e6f8-43ae-9caa-1d3485a5db5f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.351620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3498e-e6f8-43ae-9caa-1d3485a5db5f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.351912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3498e-e6f8-43ae-9caa-1d3485a5db5f None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.353098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3498e-e6f8-43ae-9caa-1d3485a5db5f None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:04.353397 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3498e-e6f8-43ae-9caa-1d3485a5db5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.354978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3498e-e6f8-43ae-9caa-1d3485a5db5f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.362892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bb952-6d1b-4d5e-902a-75f828b5c9a0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.363152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 274/572] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:04.363950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3498e-e6f8-43ae-9caa-1d3485a5db5f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.402610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd57c7-d200-46e3-afcf-f36343b00962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.403097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd57c7-d200-46e3-afcf-f36343b00962 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.403369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd57c7-d200-46e3-afcf-f36343b00962 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.403859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd57c7-d200-46e3-afcf-f36343b00962 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.404499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd57c7-d200-46e3-afcf-f36343b00962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:04.404823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd57c7-d200-46e3-afcf-f36343b00962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.406210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd57c7-d200-46e3-afcf-f36343b00962 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.414048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:04.419008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 299/573] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:04.419905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd57c7-d200-46e3-afcf-f36343b00962 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.462927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:04.469259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3498e-e6f8-43ae-9caa-1d3485a5db5f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.502971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d706834-0751-4a7c-8af4-e5d14ae5455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.503505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d706834-0751-4a7c-8af4-e5d14ae5455e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.503784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d706834-0751-4a7c-8af4-e5d14ae5455e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.504054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d706834-0751-4a7c-8af4-e5d14ae5455e None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.504691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d706834-0751-4a7c-8af4-e5d14ae5455e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:04.504995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d706834-0751-4a7c-8af4-e5d14ae5455e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.506405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d706834-0751-4a7c-8af4-e5d14ae5455e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.516690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd57c7-d200-46e3-afcf-f36343b00962 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.519120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 275/574] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:04.519918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d706834-0751-4a7c-8af4-e5d14ae5455e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.550115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ef69f-6e5e-4b8d-a9c4-73cd1a63105d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.550711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ef69f-6e5e-4b8d-a9c4-73cd1a63105d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.550988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ef69f-6e5e-4b8d-a9c4-73cd1a63105d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.551256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ef69f-6e5e-4b8d-a9c4-73cd1a63105d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.551973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ef69f-6e5e-4b8d-a9c4-73cd1a63105d None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=92130daa3c3a49c2adc597dc563b2973) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:04.552277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ef69f-6e5e-4b8d-a9c4-73cd1a63105d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.553694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ef69f-6e5e-4b8d-a9c4-73cd1a63105d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.558924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 300/575] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/role_assignments?user.id=92130daa3c3a49c2adc597dc563b2973&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:04.560050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5d8c6-d744-40f0-be32-93d784835006 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.560335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5d8c6-d744-40f0-be32-93d784835006 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.560609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5d8c6-d744-40f0-be32-93d784835006 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.561984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 301/576] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:04.562531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5d8c6-d744-40f0-be32-93d784835006 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.646179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:04.646387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d706834-0751-4a7c-8af4-e5d14ae5455e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.687630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaad0e3-28cd-432c-ab15-9b0afa4a077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.688270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaad0e3-28cd-432c-ab15-9b0afa4a077f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.688555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaad0e3-28cd-432c-ab15-9b0afa4a077f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.688827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaad0e3-28cd-432c-ab15-9b0afa4a077f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.690374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5d8c6-d744-40f0-be32-93d784835006 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.700858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaad0e3-28cd-432c-ab15-9b0afa4a077f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.702767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaad0e3-28cd-432c-ab15-9b0afa4a077f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-1eaad0e3-28cd-432c-ab15-9b0afa4a077f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:04.720292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:04.721419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:04.722525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:04.722525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:04.722525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:04.722525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 276/577] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:04.722525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaad0e3-28cd-432c-ab15-9b0afa4a077f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.731811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:04.739184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9ebdd-89e3-4c3b-b447-da6cd66cadff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.739641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9ebdd-89e3-4c3b-b447-da6cd66cadff None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.739918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9ebdd-89e3-4c3b-b447-da6cd66cadff None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.740186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9ebdd-89e3-4c3b-b447-da6cd66cadff None admin] PATH_INFO: `/v3/users/nova` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.748385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9ebdd-89e3-4c3b-b447-da6cd66cadff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.749891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9ebdd-89e3-4c3b-b447-da6cd66cadff None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-bbc9ebdd-89e3-4c3b-b447-da6cd66cadff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:04.769456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:04.770591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:04.771492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 302/578] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:04.772091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d4fe6-45fd-48e0-a551-2975e77bae3d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.772091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d4fe6-45fd-48e0-a551-2975e77bae3d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.772211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d4fe6-45fd-48e0-a551-2975e77bae3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.790787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-509d4fe6-45fd-48e0-a551-2975e77bae3d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:04.791996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-509d4fe6-45fd-48e0-a551-2975e77bae3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:04.834485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaad0e3-28cd-432c-ab15-9b0afa4a077f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:04.872745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 303/579] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:04.872745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d4fe6-45fd-48e0-a551-2975e77bae3d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.878487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d563eb1-a36e-498e-bf68-472e5a63937a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:04.879195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d563eb1-a36e-498e-bf68-472e5a63937a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:04.879571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d563eb1-a36e-498e-bf68-472e5a63937a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:04.879937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d563eb1-a36e-498e-bf68-472e5a63937a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:04.888111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d563eb1-a36e-498e-bf68-472e5a63937a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:04.889672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d563eb1-a36e-498e-bf68-472e5a63937a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-6d563eb1-a36e-498e-bf68-472e5a63937a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:04.901456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:04.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:04.903731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:04.903731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:04.903731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:04.903731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:04.903731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:04.903731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:04.903731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:04.903731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:04.903731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 277/580] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:04.904354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d563eb1-a36e-498e-bf68-472e5a63937a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:04.929129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:04.972234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:04.977200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d4fe6-45fd-48e0-a551-2975e77bae3d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.011407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d563eb1-a36e-498e-bf68-472e5a63937a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.016238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e83b8-b1ec-478c-8709-941e85f72303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.017054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e83b8-b1ec-478c-8709-941e85f72303 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.017481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e83b8-b1ec-478c-8709-941e85f72303 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.017932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e83b8-b1ec-478c-8709-941e85f72303 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.019256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e83b8-b1ec-478c-8709-941e85f72303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:05.019852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e83b8-b1ec-478c-8709-941e85f72303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.021740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e83b8-b1ec-478c-8709-941e85f72303 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.039068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 304/581] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:05.039068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e83b8-b1ec-478c-8709-941e85f72303 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.052264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb115e2-3877-4c16-ae74-ce733733189a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.052959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb115e2-3877-4c16-ae74-ce733733189a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.053240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb115e2-3877-4c16-ae74-ce733733189a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.053508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb115e2-3877-4c16-ae74-ce733733189a None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.054273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb115e2-3877-4c16-ae74-ce733733189a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:05.054577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb115e2-3877-4c16-ae74-ce733733189a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.056561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb115e2-3877-4c16-ae74-ce733733189a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.079893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 278/582] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Aug 8 16:41:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:05.079893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb115e2-3877-4c16-ae74-ce733733189a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.149118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e83b8-b1ec-478c-8709-941e85f72303 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.164373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:05.178544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb115e2-3877-4c16-ae74-ce733733189a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.187697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661afc56-29b0-4f30-b928-3adaf71d2c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.188667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661afc56-29b0-4f30-b928-3adaf71d2c06 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.189041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661afc56-29b0-4f30-b928-3adaf71d2c06 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.189309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661afc56-29b0-4f30-b928-3adaf71d2c06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.203186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661afc56-29b0-4f30-b928-3adaf71d2c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:05.204573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-661afc56-29b0-4f30-b928-3adaf71d2c06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:05.207719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:05.208656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:05.209528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 305/583] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:05.210835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661afc56-29b0-4f30-b928-3adaf71d2c06 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.225269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd52f-6190-43df-82d8-3667ee4105fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.226227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd52f-6190-43df-82d8-3667ee4105fd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.226700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd52f-6190-43df-82d8-3667ee4105fd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.227054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd52f-6190-43df-82d8-3667ee4105fd None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.228310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddd52f-6190-43df-82d8-3667ee4105fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:05.228703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddd52f-6190-43df-82d8-3667ee4105fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.230254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddd52f-6190-43df-82d8-3667ee4105fd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.247938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 279/584] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:05.248929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd52f-6190-43df-82d8-3667ee4105fd None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.318929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661afc56-29b0-4f30-b928-3adaf71d2c06 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.351138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd52f-6190-43df-82d8-3667ee4105fd None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.353776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab56d2b-32a1-492f-9a99-58ace25ac794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.354418 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab56d2b-32a1-492f-9a99-58ace25ac794 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.354695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab56d2b-32a1-492f-9a99-58ace25ac794 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.354959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab56d2b-32a1-492f-9a99-58ace25ac794 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.362397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab56d2b-32a1-492f-9a99-58ace25ac794 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.363823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab56d2b-32a1-492f-9a99-58ace25ac794 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-6ab56d2b-32a1-492f-9a99-58ace25ac794 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:05.375202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:05.376283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 306/585] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:05.382047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab56d2b-32a1-492f-9a99-58ace25ac794 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.395979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef44d4-0f89-4c35-8208-3f4d595eb838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.397248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef44d4-0f89-4c35-8208-3f4d595eb838 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.397971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef44d4-0f89-4c35-8208-3f4d595eb838 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.398383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef44d4-0f89-4c35-8208-3f4d595eb838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.412790 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:05.413939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef44d4-0f89-4c35-8208-3f4d595eb838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.415510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef44d4-0f89-4c35-8208-3f4d595eb838 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-eeef44d4-0f89-4c35-8208-3f4d595eb838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:05.430543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:05.431655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:05.432694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:05.432694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:05.432694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:05.437988 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 280/586] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:05.437988 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef44d4-0f89-4c35-8208-3f4d595eb838 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.471348 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:05.492024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab56d2b-32a1-492f-9a99-58ace25ac794 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.529412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b21d8-b74a-4243-9795-3ba2215bf4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.530377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b21d8-b74a-4243-9795-3ba2215bf4ad None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.530729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b21d8-b74a-4243-9795-3ba2215bf4ad None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.531210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b21d8-b74a-4243-9795-3ba2215bf4ad None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.532264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b21d8-b74a-4243-9795-3ba2215bf4ad None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:05.532630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b21d8-b74a-4243-9795-3ba2215bf4ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.534068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b21d8-b74a-4243-9795-3ba2215bf4ad None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.540784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef44d4-0f89-4c35-8208-3f4d595eb838 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.546605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 307/587] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:05.546605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b21d8-b74a-4243-9795-3ba2215bf4ad None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.580904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3234d-a0d1-4c64-ae47-f9d5595d1074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.581573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3234d-a0d1-4c64-ae47-f9d5595d1074 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.581959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3234d-a0d1-4c64-ae47-f9d5595d1074 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.582300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3234d-a0d1-4c64-ae47-f9d5595d1074 None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/73bfe655c6d34918b21acb8748b9156d/roles/68040d0d137e4250a000b019915a68e2` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.602491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3234d-a0d1-4c64-ae47-f9d5595d1074 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=73bfe655c6d34918b21acb8748b9156d, role_id=68040d0d137e4250a000b019915a68e2)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.604671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3234d-a0d1-4c64-ae47-f9d5595d1074 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.647573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:05.654425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b21d8-b74a-4243-9795-3ba2215bf4ad None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.685829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 281/588] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:41:05 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/73bfe655c6d34918b21acb8748b9156d/roles/68040d0d137e4250a000b019915a68e2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:41:05.685829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3234d-a0d1-4c64-ae47-f9d5595d1074 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.702465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfe3df-9bda-488f-b943-dc42f680adb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.704568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfe3df-9bda-488f-b943-dc42f680adb2 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.704568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfe3df-9bda-488f-b943-dc42f680adb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.704568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfe3df-9bda-488f-b943-dc42f680adb2 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.705599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfe3df-9bda-488f-b943-dc42f680adb2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:05.705999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfe3df-9bda-488f-b943-dc42f680adb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.708329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfe3df-9bda-488f-b943-dc42f680adb2 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.734107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 308/589] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:05.735175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfe3df-9bda-488f-b943-dc42f680adb2 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.804690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:05.823392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3234d-a0d1-4c64-ae47-f9d5595d1074 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.865034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0b94d-8283-48c4-9775-33bdccc5c484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.866048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0b94d-8283-48c4-9775-33bdccc5c484 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.866506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0b94d-8283-48c4-9775-33bdccc5c484 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.866951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0b94d-8283-48c4-9775-33bdccc5c484 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.867849 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0b94d-8283-48c4-9775-33bdccc5c484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:05.868899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0b94d-8283-48c4-9775-33bdccc5c484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.870633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0b94d-8283-48c4-9775-33bdccc5c484 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.876413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfe3df-9bda-488f-b943-dc42f680adb2 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:05.884744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 282/590] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:05.886167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0b94d-8283-48c4-9775-33bdccc5c484 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.919650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743da7e-04e3-4ef1-a051-66c34acad335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:05.920584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743da7e-04e3-4ef1-a051-66c34acad335 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:05.921038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743da7e-04e3-4ef1-a051-66c34acad335 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:05.921499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743da7e-04e3-4ef1-a051-66c34acad335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:05.932921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743da7e-04e3-4ef1-a051-66c34acad335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:05.934620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743da7e-04e3-4ef1-a051-66c34acad335 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-0743da7e-04e3-4ef1-a051-66c34acad335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:05.950192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:05.951156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:05.952007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:05.952007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:05.952007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:05.954111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 309/591] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:05.955234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743da7e-04e3-4ef1-a051-66c34acad335 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:05.960094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:05.999286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0b94d-8283-48c4-9775-33bdccc5c484 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.037098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76cdf6-a906-4810-951c-38aef61952fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.037770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76cdf6-a906-4810-951c-38aef61952fe None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.038075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76cdf6-a906-4810-951c-38aef61952fe None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.038343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76cdf6-a906-4810-951c-38aef61952fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.050810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76cdf6-a906-4810-951c-38aef61952fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.052254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76cdf6-a906-4810-951c-38aef61952fe None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.059925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:06.068863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743da7e-04e3-4ef1-a051-66c34acad335 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-9c76cdf6-a906-4810-951c-38aef61952fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:06.069488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:06.071463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:06.072526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:06.072526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:06.072526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:06.078270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 283/592] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:06.079338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76cdf6-a906-4810-951c-38aef61952fe None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:06.104364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807caed-88d2-4e80-b2c5-ae1d703865ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.104995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807caed-88d2-4e80-b2c5-ae1d703865ab None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.105274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807caed-88d2-4e80-b2c5-ae1d703865ab None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.105541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807caed-88d2-4e80-b2c5-ae1d703865ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.112564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807caed-88d2-4e80-b2c5-ae1d703865ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.113943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807caed-88d2-4e80-b2c5-ae1d703865ab None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-4807caed-88d2-4e80-b2c5-ae1d703865ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:06.126019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:06.127082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:06.128085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:06.128085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:06.128085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:06.128085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:06.128085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:06.128085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:06.128085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:06.128085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:06.128085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 310/593] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:06.128584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807caed-88d2-4e80-b2c5-ae1d703865ab None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:06.194249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76cdf6-a906-4810-951c-38aef61952fe None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.198338 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:06.233190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9fd79-8140-428b-a1a5-d89caef6cf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.234065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9fd79-8140-428b-a1a5-d89caef6cf2d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.234482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9fd79-8140-428b-a1a5-d89caef6cf2d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.234890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9fd79-8140-428b-a1a5-d89caef6cf2d None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.235749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9fd79-8140-428b-a1a5-d89caef6cf2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:06.236258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9fd79-8140-428b-a1a5-d89caef6cf2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.237755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9fd79-8140-428b-a1a5-d89caef6cf2d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.238589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807caed-88d2-4e80-b2c5-ae1d703865ab None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.250899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 284/594] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:06.252007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9fd79-8140-428b-a1a5-d89caef6cf2d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:06.273571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ef96e-3480-4483-9277-ebc37d24a375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.274793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ef96e-3480-4483-9277-ebc37d24a375 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.275069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ef96e-3480-4483-9277-ebc37d24a375 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.275333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ef96e-3480-4483-9277-ebc37d24a375 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.276282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ef96e-3480-4483-9277-ebc37d24a375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:06.276282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ef96e-3480-4483-9277-ebc37d24a375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.277616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ef96e-3480-4483-9277-ebc37d24a375 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.294044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 311/595] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:06.295017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ef96e-3480-4483-9277-ebc37d24a375 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:06.318031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:06.364077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9fd79-8140-428b-a1a5-d89caef6cf2d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.401693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3533b-ecab-4c8b-ad4a-5a473d1b82d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.402251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ef96e-3480-4483-9277-ebc37d24a375 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.402805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3533b-ecab-4c8b-ad4a-5a473d1b82d5 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.403248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3533b-ecab-4c8b-ad4a-5a473d1b82d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.403785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3533b-ecab-4c8b-ad4a-5a473d1b82d5 None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.404616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3533b-ecab-4c8b-ad4a-5a473d1b82d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:06.405067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3533b-ecab-4c8b-ad4a-5a473d1b82d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.406657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3533b-ecab-4c8b-ad4a-5a473d1b82d5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.419931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 285/596] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:06.421072 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3533b-ecab-4c8b-ad4a-5a473d1b82d5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:06.433235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:06.444630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fd2b1-edcf-4b11-97da-3892fcd72a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.445396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fd2b1-edcf-4b11-97da-3892fcd72a58 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.445938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fd2b1-edcf-4b11-97da-3892fcd72a58 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.446771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fd2b1-edcf-4b11-97da-3892fcd72a58 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.454395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fd2b1-edcf-4b11-97da-3892fcd72a58 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.455952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fd2b1-edcf-4b11-97da-3892fcd72a58 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-d45fd2b1-edcf-4b11-97da-3892fcd72a58 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:06.468251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:06.469388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:41:06.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:06.472346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 312/597] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:06.473187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fd2b1-edcf-4b11-97da-3892fcd72a58 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:06.539342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3533b-ecab-4c8b-ad4a-5a473d1b82d5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.557941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:06.576073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658d31b-7792-45a0-9339-7376ad4685a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.576703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658d31b-7792-45a0-9339-7376ad4685a7 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.577552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fd2b1-edcf-4b11-97da-3892fcd72a58 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.577651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658d31b-7792-45a0-9339-7376ad4685a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.577651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658d31b-7792-45a0-9339-7376ad4685a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.586396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658d31b-7792-45a0-9339-7376ad4685a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.588515 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658d31b-7792-45a0-9339-7376ad4685a7 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-1658d31b-7792-45a0-9339-7376ad4685a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:06.604466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:06.605502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:06.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:06.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:06.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:06.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:06.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:06.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:06.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:06.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:06.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 286/598] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:06.607048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658d31b-7792-45a0-9339-7376ad4685a7 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:06.621283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f45e7-88c4-4d6c-ac46-416a94974407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.622205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f45e7-88c4-4d6c-ac46-416a94974407 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.622692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f45e7-88c4-4d6c-ac46-416a94974407 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.623192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f45e7-88c4-4d6c-ac46-416a94974407 None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/54b05cbc41b844d8932ba4c15d960289/roles/68040d0d137e4250a000b019915a68e2` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.641920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f45e7-88c4-4d6c-ac46-416a94974407 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=54b05cbc41b844d8932ba4c15d960289, role_id=68040d0d137e4250a000b019915a68e2)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.643422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f45e7-88c4-4d6c-ac46-416a94974407 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.706690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658d31b-7792-45a0-9339-7376ad4685a7 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.722133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 313/599] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:41:06 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/54b05cbc41b844d8932ba4c15d960289/roles/68040d0d137e4250a000b019915a68e2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:41:06.723592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba57a0a-fe2f-49f6-8fdc-1941a1f7b595 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.723937 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba57a0a-fe2f-49f6-8fdc-1941a1f7b595 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.724209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba57a0a-fe2f-49f6-8fdc-1941a1f7b595 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.730610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 314/600] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:06.731640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba57a0a-fe2f-49f6-8fdc-1941a1f7b595 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:06.753881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebafa4-4c6f-4c56-8721-18d4bd60e3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.754890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebafa4-4c6f-4c56-8721-18d4bd60e3a6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.755313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebafa4-4c6f-4c56-8721-18d4bd60e3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.755762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebafa4-4c6f-4c56-8721-18d4bd60e3a6 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.757578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebafa4-4c6f-4c56-8721-18d4bd60e3a6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:06.758426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebafa4-4c6f-4c56-8721-18d4bd60e3a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.761185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebafa4-4c6f-4c56-8721-18d4bd60e3a6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.772631 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:06.787600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 287/601] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Aug 8 16:41:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:06.790720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820ccf-a791-4a36-92df-32b58c8d7ab6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.791212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820ccf-a791-4a36-92df-32b58c8d7ab6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.791816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18820ccf-a791-4a36-92df-32b58c8d7ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.816966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-18820ccf-a791-4a36-92df-32b58c8d7ab6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:06.819682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-18820ccf-a791-4a36-92df-32b58c8d7ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:06.859888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba57a0a-fe2f-49f6-8fdc-1941a1f7b595 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:06.867080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:06.905037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b8c3a-3980-43f6-8eae-9b0cd6c6ef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:06.905826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b8c3a-3980-43f6-8eae-9b0cd6c6ef66 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:06.905956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b8c3a-3980-43f6-8eae-9b0cd6c6ef66 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:06.906204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b8c3a-3980-43f6-8eae-9b0cd6c6ef66 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:06.906898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b8c3a-3980-43f6-8eae-9b0cd6c6ef66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:06.907198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b8c3a-3980-43f6-8eae-9b0cd6c6ef66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:06.908622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b8c3a-3980-43f6-8eae-9b0cd6c6ef66 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:06.920394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 288/602] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:06.921833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18820ccf-a791-4a36-92df-32b58c8d7ab6 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:06.924244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 315/603] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:06.929991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b8c3a-3980-43f6-8eae-9b0cd6c6ef66 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:07.036371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:07.043583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b8c3a-3980-43f6-8eae-9b0cd6c6ef66 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:07.063467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18820ccf-a791-4a36-92df-32b58c8d7ab6 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:07.081861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6a7e7-6a2f-4c69-b4ac-c2dc7dd55655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:07.082902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6a7e7-6a2f-4c69-b4ac-c2dc7dd55655 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:07.083376 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6a7e7-6a2f-4c69-b4ac-c2dc7dd55655 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:07.083885 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6a7e7-6a2f-4c69-b4ac-c2dc7dd55655 None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/e860aad24688491c9a89ddfb35922eb4/roles/68040d0d137e4250a000b019915a68e2` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:07.099384 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6a7e7-6a2f-4c69-b4ac-c2dc7dd55655 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=e860aad24688491c9a89ddfb35922eb4, role_id=68040d0d137e4250a000b019915a68e2)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:07.101157 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6a7e7-6a2f-4c69-b4ac-c2dc7dd55655 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:07.107953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268ae4e-8306-485b-b1c1-5e049890b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:07.108904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268ae4e-8306-485b-b1c1-5e049890b83a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:07.109437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268ae4e-8306-485b-b1c1-5e049890b83a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:07.109943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268ae4e-8306-485b-b1c1-5e049890b83a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:07.122287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268ae4e-8306-485b-b1c1-5e049890b83a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:07.124066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268ae4e-8306-485b-b1c1-5e049890b83a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-9268ae4e-8306-485b-b1c1-5e049890b83a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:07.140312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:07.141264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:07.147884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:07.147884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:07.147884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:07.147884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 289/604] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:07.147884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268ae4e-8306-485b-b1c1-5e049890b83a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:07.163604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:07.193012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 316/605] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:41:06 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/e860aad24688491c9a89ddfb35922eb4/roles/68040d0d137e4250a000b019915a68e2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 16:41:07.194459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6a7e7-6a2f-4c69-b4ac-c2dc7dd55655 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:07.277456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268ae4e-8306-485b-b1c1-5e049890b83a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:07.321903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6a7e7-6a2f-4c69-b4ac-c2dc7dd55655 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:07.362122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ebf52-fa3a-4da4-a373-5b675e586f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:07.362740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ebf52-fa3a-4da4-a373-5b675e586f46 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:07.363016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ebf52-fa3a-4da4-a373-5b675e586f46 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:07.363284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ebf52-fa3a-4da4-a373-5b675e586f46 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:07.363935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ebf52-fa3a-4da4-a373-5b675e586f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:07.364235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ebf52-fa3a-4da4-a373-5b675e586f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:07.365181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697957f-18ce-434b-b652-7991936a44af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:07.365611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ebf52-fa3a-4da4-a373-5b675e586f46 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:07.365848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697957f-18ce-434b-b652-7991936a44af None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:07.366126 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697957f-18ce-434b-b652-7991936a44af None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:07.366393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697957f-18ce-434b-b652-7991936a44af None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:07.379127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 317/606] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:07.384994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ebf52-fa3a-4da4-a373-5b675e586f46 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:07.392240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:07.396462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697957f-18ce-434b-b652-7991936a44af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b697957f-18ce-434b-b652-7991936a44af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:07.403427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:07.410635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:07.411480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 290/607] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:07.414163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697957f-18ce-434b-b652-7991936a44af None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:07.546278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ebf52-fa3a-4da4-a373-5b675e586f46 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:07.549512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:07.593120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697957f-18ce-434b-b652-7991936a44af None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:07.614154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea63dd-6835-4046-bc79-9d2df52f5bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:07.614786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea63dd-6835-4046-bc79-9d2df52f5bed None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:07.615052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea63dd-6835-4046-bc79-9d2df52f5bed None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:07.615315 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea63dd-6835-4046-bc79-9d2df52f5bed None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:07.629071 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea63dd-6835-4046-bc79-9d2df52f5bed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:07.634577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea63dd-6835-4046-bc79-9d2df52f5bed None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-6dea63dd-6835-4046-bc79-9d2df52f5bed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:07.655409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:07.656309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:07.657190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:07.657190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:07.657190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:07.657190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:07.657190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:07.657190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:07.657190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:07.657190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:07.662186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 318/608] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:07.663892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49055722-eee3-466e-8ccf-9a50f18e2e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:07.665342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49055722-eee3-466e-8ccf-9a50f18e2e7e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:07.666436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea63dd-6835-4046-bc79-9d2df52f5bed None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:07.666967 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49055722-eee3-466e-8ccf-9a50f18e2e7e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:07.667456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49055722-eee3-466e-8ccf-9a50f18e2e7e None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:07.668807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49055722-eee3-466e-8ccf-9a50f18e2e7e None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:07.670231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49055722-eee3-466e-8ccf-9a50f18e2e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:07.671807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49055722-eee3-466e-8ccf-9a50f18e2e7e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:07.692482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 291/609] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:07.698192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49055722-eee3-466e-8ccf-9a50f18e2e7e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:07.791079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:07.867212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea63dd-6835-4046-bc79-9d2df52f5bed None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:07.881839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49055722-eee3-466e-8ccf-9a50f18e2e7e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:07.921068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c85ad-5be0-4669-836d-886342f4f9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:07.921701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c85ad-5be0-4669-836d-886342f4f9e3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:07.921999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c85ad-5be0-4669-836d-886342f4f9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:07.922266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c85ad-5be0-4669-836d-886342f4f9e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:07.932878 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:07.936160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c85ad-5be0-4669-836d-886342f4f9e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:07.938007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c85ad-5be0-4669-836d-886342f4f9e3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:07.955364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804e150-2dcc-4918-a5c6-faac2cfd7da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-143c85ad-5be0-4669-836d-886342f4f9e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:07.956544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:07.957457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:07.958595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:07.958595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:07.958595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:07.958595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 292/610] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:07.959305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e150-2dcc-4918-a5c6-faac2cfd7da7 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:07.959612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e150-2dcc-4918-a5c6-faac2cfd7da7 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:07.960019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e150-2dcc-4918-a5c6-faac2cfd7da7 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:07.960752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804e150-2dcc-4918-a5c6-faac2cfd7da7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:07.960957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804e150-2dcc-4918-a5c6-faac2cfd7da7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:07.969992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804e150-2dcc-4918-a5c6-faac2cfd7da7 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:07.970160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c85ad-5be0-4669-836d-886342f4f9e3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:07.982047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 319/611] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:07.983544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe11d-ef3c-469b-a959-a61b33065bf3 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:07.983956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe11d-ef3c-469b-a959-a61b33065bf3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:07.984431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe11d-ef3c-469b-a959-a61b33065bf3 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:07.989841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 320/612] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:07.991544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe11d-ef3c-469b-a959-a61b33065bf3 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:08.102894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe11d-ef3c-469b-a959-a61b33065bf3 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:08.129436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:08.135985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c85ad-5be0-4669-836d-886342f4f9e3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:08.148877 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f208-5c59-4eb6-9413-89c0f9329f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:08.149527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f208-5c59-4eb6-9413-89c0f9329f42 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:08.149814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f208-5c59-4eb6-9413-89c0f9329f42 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:08.150148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f208-5c59-4eb6-9413-89c0f9329f42 None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/f50ee57e2418417cbe68915ca3f00f76/roles/68040d0d137e4250a000b019915a68e2` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:08.178783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2f208-5c59-4eb6-9413-89c0f9329f42 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=f50ee57e2418417cbe68915ca3f00f76, role_id=68040d0d137e4250a000b019915a68e2)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:08.181388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2f208-5c59-4eb6-9413-89c0f9329f42 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:08.195384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbf93f-be9e-408b-bdbd-e2823164549d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:08.196067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbf93f-be9e-408b-bdbd-e2823164549d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:08.196343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbf93f-be9e-408b-bdbd-e2823164549d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:08.196606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbf93f-be9e-408b-bdbd-e2823164549d None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:08.197342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbf93f-be9e-408b-bdbd-e2823164549d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:08.197645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbf93f-be9e-408b-bdbd-e2823164549d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:08.199620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbf93f-be9e-408b-bdbd-e2823164549d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:08.212632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 293/613] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:08.213980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f215327-dcee-44f9-bce1-5183f5d588ca None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:08.214313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f215327-dcee-44f9-bce1-5183f5d588ca None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:08.214582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f215327-dcee-44f9-bce1-5183f5d588ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:08.236810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3f215327-dcee-44f9-bce1-5183f5d588ca None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:08.238022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3f215327-dcee-44f9-bce1-5183f5d588ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:08.252751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 321/614] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:41:07 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/f50ee57e2418417cbe68915ca3f00f76/roles/68040d0d137e4250a000b019915a68e2 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:41:08.253553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f208-5c59-4eb6-9413-89c0f9329f42 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:08.384554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:08.456165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 294/615] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:08.481263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f208-5c59-4eb6-9413-89c0f9329f42 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:08.518538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f215327-dcee-44f9-bce1-5183f5d588ca None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:08.521415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf05ec0-db68-4ba8-a9a1-4cfdab608544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:08.522308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf05ec0-db68-4ba8-a9a1-4cfdab608544 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:08.522780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf05ec0-db68-4ba8-a9a1-4cfdab608544 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:08.523239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf05ec0-db68-4ba8-a9a1-4cfdab608544 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:08.533009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf05ec0-db68-4ba8-a9a1-4cfdab608544 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:08.534795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf05ec0-db68-4ba8-a9a1-4cfdab608544 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-ebf05ec0-db68-4ba8-a9a1-4cfdab608544 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:08.548440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:08.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:08.550369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:08.553168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 322/616] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:08.559458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf05ec0-db68-4ba8-a9a1-4cfdab608544 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:08.653344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:08.730533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f215327-dcee-44f9-bce1-5183f5d588ca None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:08.777827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf05ec0-db68-4ba8-a9a1-4cfdab608544 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:08.801200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89db6b-cbfa-4184-997b-d50d52974552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:08.801865 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89db6b-cbfa-4184-997b-d50d52974552 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:08.802143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89db6b-cbfa-4184-997b-d50d52974552 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:08.802411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89db6b-cbfa-4184-997b-d50d52974552 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:08.813719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19617b-398d-4efb-91f0-06d8e4387c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:08.814359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19617b-398d-4efb-91f0-06d8e4387c4b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:08.814637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19617b-398d-4efb-91f0-06d8e4387c4b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:08.814909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19617b-398d-4efb-91f0-06d8e4387c4b None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:08.815633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19617b-398d-4efb-91f0-06d8e4387c4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:08.815940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19617b-398d-4efb-91f0-06d8e4387c4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:08.817497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19617b-398d-4efb-91f0-06d8e4387c4b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:08.825522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89db6b-cbfa-4184-997b-d50d52974552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-ca89db6b-cbfa-4184-997b-d50d52974552 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:08.830225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:08.831114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:08.831987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 295/617] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:08.841269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 323/618] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Aug 8 16:41:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:08.842280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89db6b-cbfa-4184-997b-d50d52974552 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:08.846014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19617b-398d-4efb-91f0-06d8e4387c4b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:08.939989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:08.989205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89db6b-cbfa-4184-997b-d50d52974552 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.022910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:09.031886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19617b-398d-4efb-91f0-06d8e4387c4b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.034034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4a795-46fe-4720-9863-68585cc5149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.035496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4a795-46fe-4720-9863-68585cc5149b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.035778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4a795-46fe-4720-9863-68585cc5149b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.036055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4a795-46fe-4720-9863-68585cc5149b None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.036726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4a795-46fe-4720-9863-68585cc5149b None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:09.037029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4a795-46fe-4720-9863-68585cc5149b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:09.038550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4a795-46fe-4720-9863-68585cc5149b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:09.050630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 296/619] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:09.051919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66694294-869f-4409-8c68-87d795f043f9 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.052208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66694294-869f-4409-8c68-87d795f043f9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.052484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66694294-869f-4409-8c68-87d795f043f9 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.053955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 297/620] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:09.062733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66694294-869f-4409-8c68-87d795f043f9 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:09.078457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b9880-36dc-40e0-9b75-29508e569d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.079083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b9880-36dc-40e0-9b75-29508e569d7c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.079356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b9880-36dc-40e0-9b75-29508e569d7c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.079620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b9880-36dc-40e0-9b75-29508e569d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.090317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b9880-36dc-40e0-9b75-29508e569d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:09.091770 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b9880-36dc-40e0-9b75-29508e569d7c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-8e9b9880-36dc-40e0-9b75-29508e569d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:09.107749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:09.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:09.109579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:09.109579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:09.109579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:09.113455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 324/621] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:09.114719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2499954-d650-4133-b05d-457a0e25f7d9 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.115001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2499954-d650-4133-b05d-457a0e25f7d9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.115270 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2499954-d650-4133-b05d-457a0e25f7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.136065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d2499954-d650-4133-b05d-457a0e25f7d9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:09.137251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d2499954-d650-4133-b05d-457a0e25f7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:09.192998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66694294-869f-4409-8c68-87d795f043f9 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.196479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:09.213774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 325/622] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:09.215959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2499954-d650-4133-b05d-457a0e25f7d9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:09.240784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c2d79-adef-46c9-a01f-cad8b297698d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.241999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c2d79-adef-46c9-a01f-cad8b297698d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.242277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c2d79-adef-46c9-a01f-cad8b297698d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.245809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c2d79-adef-46c9-a01f-cad8b297698d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.257586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c2d79-adef-46c9-a01f-cad8b297698d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:09.259774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c2d79-adef-46c9-a01f-cad8b297698d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-303c2d79-adef-46c9-a01f-cad8b297698d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:09.280645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:09.282533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:09.284062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:09.284062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:09.284062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:09.284062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 298/623] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:09.284062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c2d79-adef-46c9-a01f-cad8b297698d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:09.310341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:09.348552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2499954-d650-4133-b05d-457a0e25f7d9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.396575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd2ecc-5c6b-455c-9a58-39893c5b39e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.397825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd2ecc-5c6b-455c-9a58-39893c5b39e0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.398309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd2ecc-5c6b-455c-9a58-39893c5b39e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.399375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd2ecc-5c6b-455c-9a58-39893c5b39e0 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.399588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c2d79-adef-46c9-a01f-cad8b297698d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.400836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd2ecc-5c6b-455c-9a58-39893c5b39e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:09.401614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd2ecc-5c6b-455c-9a58-39893c5b39e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:09.404257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd2ecc-5c6b-455c-9a58-39893c5b39e0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:09.421535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 326/624] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:09.422659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd2ecc-5c6b-455c-9a58-39893c5b39e0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:09.440642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c263e-e70f-4dc3-92c1-7133da3340af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.441337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c263e-e70f-4dc3-92c1-7133da3340af None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.441608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c263e-e70f-4dc3-92c1-7133da3340af None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.441915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c263e-e70f-4dc3-92c1-7133da3340af None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.453033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c263e-e70f-4dc3-92c1-7133da3340af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-7d4c263e-e70f-4dc3-92c1-7133da3340af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:09.458065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:09.458974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:09.459792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:09.464572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 299/625] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:09.465402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c263e-e70f-4dc3-92c1-7133da3340af None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:09.486063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:09.553995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd2ecc-5c6b-455c-9a58-39893c5b39e0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.596770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c263e-e70f-4dc3-92c1-7133da3340af None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.597898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642a469-9674-4651-b9ac-f0d4fd6a8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.598668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642a469-9674-4651-b9ac-f0d4fd6a8c8e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.599082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642a469-9674-4651-b9ac-f0d4fd6a8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.599510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642a469-9674-4651-b9ac-f0d4fd6a8c8e None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.600300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642a469-9674-4651-b9ac-f0d4fd6a8c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:09.600739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642a469-9674-4651-b9ac-f0d4fd6a8c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:09.602848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642a469-9674-4651-b9ac-f0d4fd6a8c8e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:09.604796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:09.615867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 327/626] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:09.616979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642a469-9674-4651-b9ac-f0d4fd6a8c8e None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:09.638953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242000c-dd7c-48cf-b9a0-f028501e748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.639565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242000c-dd7c-48cf-b9a0-f028501e748b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.640200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242000c-dd7c-48cf-b9a0-f028501e748b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.640200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242000c-dd7c-48cf-b9a0-f028501e748b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.646776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242000c-dd7c-48cf-b9a0-f028501e748b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:09.648246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242000c-dd7c-48cf-b9a0-f028501e748b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-5242000c-dd7c-48cf-b9a0-f028501e748b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:09.660156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:09.661255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:09.662377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:09.662377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:09.662377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:09.662377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:09.662377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:09.662377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:09.662377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:09.662377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:09.665848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 300/627] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:09.667101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da74061-edbe-4499-944c-7be5cfc33d1d None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.667384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da74061-edbe-4499-944c-7be5cfc33d1d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.667655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da74061-edbe-4499-944c-7be5cfc33d1d None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.669827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 301/628] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:09.669827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da74061-edbe-4499-944c-7be5cfc33d1d None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:09.730341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642a469-9674-4651-b9ac-f0d4fd6a8c8e None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.754656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:09.765855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf188aef-d200-4495-a460-aa5614493528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.766593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf188aef-d200-4495-a460-aa5614493528 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.766890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf188aef-d200-4495-a460-aa5614493528 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.767223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf188aef-d200-4495-a460-aa5614493528 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.767940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf188aef-d200-4495-a460-aa5614493528 None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:09.768303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf188aef-d200-4495-a460-aa5614493528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:09.769720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf188aef-d200-4495-a460-aa5614493528 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:09.778326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 328/629] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:09.779111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf188aef-d200-4495-a460-aa5614493528 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:09.779897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da74061-edbe-4499-944c-7be5cfc33d1d None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.815770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507648d9-9d65-421a-9b40-56db19451a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.816480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507648d9-9d65-421a-9b40-56db19451a3b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.816762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507648d9-9d65-421a-9b40-56db19451a3b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.817087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507648d9-9d65-421a-9b40-56db19451a3b None admin] PATH_INFO: `/v3/users/glance` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.825502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507648d9-9d65-421a-9b40-56db19451a3b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:09.826970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507648d9-9d65-421a-9b40-56db19451a3b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-507648d9-9d65-421a-9b40-56db19451a3b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:09.839558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:09.845212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 302/630] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aafc77-ff70-45da-a09b-f89141cc2959 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aafc77-ff70-45da-a09b-f89141cc2959 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.847319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aafc77-ff70-45da-a09b-f89141cc2959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.849889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:09.864000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-52aafc77-ff70-45da-a09b-f89141cc2959 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:09.865240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-52aafc77-ff70-45da-a09b-f89141cc2959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:09.899502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf188aef-d200-4495-a460-aa5614493528 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:09.935345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 303/631] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:09.936456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aafc77-ff70-45da-a09b-f89141cc2959 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:09.952498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc56b7-370f-4531-9276-9db64f35d218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:09.953892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc56b7-370f-4531-9276-9db64f35d218 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:09.954400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc56b7-370f-4531-9276-9db64f35d218 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:09.954908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc56b7-370f-4531-9276-9db64f35d218 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:09.955738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc56b7-370f-4531-9276-9db64f35d218 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:09.956393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc56b7-370f-4531-9276-9db64f35d218 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:09.958473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc56b7-370f-4531-9276-9db64f35d218 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:09.972354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 329/632] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:09.973336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc56b7-370f-4531-9276-9db64f35d218 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:10.016393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:10.051969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aafc77-ff70-45da-a09b-f89141cc2959 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:10.078169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc56b7-370f-4531-9276-9db64f35d218 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:10.092241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c56e83-ad01-4b08-974e-c88e169fd92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:10.092863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c56e83-ad01-4b08-974e-c88e169fd92e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.093140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c56e83-ad01-4b08-974e-c88e169fd92e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.093411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c56e83-ad01-4b08-974e-c88e169fd92e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.095039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:10.104911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c56e83-ad01-4b08-974e-c88e169fd92e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:10.106397 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c56e83-ad01-4b08-974e-c88e169fd92e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:10.113335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db923705-d9c7-42fd-9a75-afab05918219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:10.113952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db923705-d9c7-42fd-9a75-afab05918219 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.114230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db923705-d9c7-42fd-9a75-afab05918219 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.114494 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db923705-d9c7-42fd-9a75-afab05918219 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.115328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db923705-d9c7-42fd-9a75-afab05918219 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:10.115626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db923705-d9c7-42fd-9a75-afab05918219 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:10.117020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db923705-d9c7-42fd-9a75-afab05918219 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-93c56e83-ad01-4b08-974e-c88e169fd92e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:10.121389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:10.122347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:10.123810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:10.123810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:10.123810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:10.123810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 304/633] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:10.123810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c56e83-ad01-4b08-974e-c88e169fd92e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:10.144645 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 330/634] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Aug 8 16:41:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:10.145664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db923705-d9c7-42fd-9a75-afab05918219 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:10.238452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c56e83-ad01-4b08-974e-c88e169fd92e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:10.245852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db923705-d9c7-42fd-9a75-afab05918219 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:10.256273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:10.276505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba907b6b-4fd4-42bf-80c5-a8848ce79784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:10.277047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba907b6b-4fd4-42bf-80c5-a8848ce79784 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.277337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba907b6b-4fd4-42bf-80c5-a8848ce79784 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.277606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba907b6b-4fd4-42bf-80c5-a8848ce79784 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.281889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e66d50-073b-46c7-b6ee-187c6de5758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:10.282541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e66d50-073b-46c7-b6ee-187c6de5758c None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.282816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e66d50-073b-46c7-b6ee-187c6de5758c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.283273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e66d50-073b-46c7-b6ee-187c6de5758c None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/92130daa3c3a49c2adc597dc563b2973/roles/68040d0d137e4250a000b019915a68e2` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.289531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba907b6b-4fd4-42bf-80c5-a8848ce79784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-ba907b6b-4fd4-42bf-80c5-a8848ce79784 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:10.294727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:10.295853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:10.296807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 305/635] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:10.298601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba907b6b-4fd4-42bf-80c5-a8848ce79784 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:10.302998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e66d50-073b-46c7-b6ee-187c6de5758c None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=92130daa3c3a49c2adc597dc563b2973, role_id=68040d0d137e4250a000b019915a68e2)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:10.304700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e66d50-073b-46c7-b6ee-187c6de5758c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:10.360507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 331/636] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:41:10 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/92130daa3c3a49c2adc597dc563b2973/roles/68040d0d137e4250a000b019915a68e2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:41:10.361758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e66d50-073b-46c7-b6ee-187c6de5758c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:10.395947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:10.431105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba907b6b-4fd4-42bf-80c5-a8848ce79784 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:10.477989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb78fdc-fc7b-4087-8e86-d352fd70b8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:10.478641 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb78fdc-fc7b-4087-8e86-d352fd70b8fa None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.478921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb78fdc-fc7b-4087-8e86-d352fd70b8fa None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.479190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb78fdc-fc7b-4087-8e86-d352fd70b8fa None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.479882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb78fdc-fc7b-4087-8e86-d352fd70b8fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:10.480188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb78fdc-fc7b-4087-8e86-d352fd70b8fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:10.481578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb78fdc-fc7b-4087-8e86-d352fd70b8fa None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:10.486503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e66d50-073b-46c7-b6ee-187c6de5758c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:10.495798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 306/637] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:10.496568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb78fdc-fc7b-4087-8e86-d352fd70b8fa None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:10.532674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0f283-2686-43ec-ad13-6e3cbea124ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:10.533315 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0f283-2686-43ec-ad13-6e3cbea124ad None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.533661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0f283-2686-43ec-ad13-6e3cbea124ad None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.534177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0f283-2686-43ec-ad13-6e3cbea124ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.546471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:10.613248 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0f283-2686-43ec-ad13-6e3cbea124ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:10.614723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0f283-2686-43ec-ad13-6e3cbea124ad None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:10.617016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb78fdc-fc7b-4087-8e86-d352fd70b8fa None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:10.640693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:10.655204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fcf61f-516d-40af-a74f-02e1013367e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:10.655891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcf61f-516d-40af-a74f-02e1013367e0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.656238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcf61f-516d-40af-a74f-02e1013367e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.656576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcf61f-516d-40af-a74f-02e1013367e0 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.657335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcf61f-516d-40af-a74f-02e1013367e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:10.657712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcf61f-516d-40af-a74f-02e1013367e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:10.659162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcf61f-516d-40af-a74f-02e1013367e0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:10.672625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 307/638] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:10.674376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e5f65-f6cb-4abb-a813-84b35c73b8af None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.674994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e5f65-f6cb-4abb-a813-84b35c73b8af None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.674994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e5f65-f6cb-4abb-a813-84b35c73b8af None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.680310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 308/639] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:10.680310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e5f65-f6cb-4abb-a813-84b35c73b8af None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:10.731360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 332/640] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:41:10.731360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0f283-2686-43ec-ad13-6e3cbea124ad None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:10.788196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:10.792087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e5f65-f6cb-4abb-a813-84b35c73b8af None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:10.827490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633e8ed-e146-4a75-9581-b78758b81391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:10.828118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e8ed-e146-4a75-9581-b78758b81391 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.828401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e8ed-e146-4a75-9581-b78758b81391 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.828671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e8ed-e146-4a75-9581-b78758b81391 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.837119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633e8ed-e146-4a75-9581-b78758b81391 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:10.838591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633e8ed-e146-4a75-9581-b78758b81391 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:10.843585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0f283-2686-43ec-ad13-6e3cbea124ad None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-0633e8ed-e146-4a75-9581-b78758b81391 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:10.849557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:10.850709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 309/641] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Aug 8 16:41:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:10.852714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68030064-28de-4a32-9c4a-3b0dd8027dd3 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.853524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68030064-28de-4a32-9c4a-3b0dd8027dd3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.853524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68030064-28de-4a32-9c4a-3b0dd8027dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.870525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-68030064-28de-4a32-9c4a-3b0dd8027dd3 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:10.871724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-68030064-28de-4a32-9c4a-3b0dd8027dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:10.872978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:10.883711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfaaf0f-b350-44c8-83e6-d24606ae5dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:10.884409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfaaf0f-b350-44c8-83e6-d24606ae5dbe None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:10.885223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfaaf0f-b350-44c8-83e6-d24606ae5dbe None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:10.885223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfaaf0f-b350-44c8-83e6-d24606ae5dbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:10.896495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfaaf0f-b350-44c8-83e6-d24606ae5dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:10.899091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfaaf0f-b350-44c8-83e6-d24606ae5dbe None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-8bfaaf0f-b350-44c8-83e6-d24606ae5dbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:10.913777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:10.915499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:10.916382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:10.916382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:10.916382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:10.916382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 333/642] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:10.917955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfaaf0f-b350-44c8-83e6-d24606ae5dbe None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:10.949584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 310/643] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:10.950764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68030064-28de-4a32-9c4a-3b0dd8027dd3 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.038876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfaaf0f-b350-44c8-83e6-d24606ae5dbe None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.052009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:11.062252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68030064-28de-4a32-9c4a-3b0dd8027dd3 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.074130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10aff-aacd-4f86-af33-e4e5bdbc2886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.074758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10aff-aacd-4f86-af33-e4e5bdbc2886 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.075033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10aff-aacd-4f86-af33-e4e5bdbc2886 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.075296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10aff-aacd-4f86-af33-e4e5bdbc2886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.086993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e10aff-aacd-4f86-af33-e4e5bdbc2886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.088314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e10aff-aacd-4f86-af33-e4e5bdbc2886 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:11.100558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12f674-e107-4a7c-865d-019cd1a01fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.101165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12f674-e107-4a7c-865d-019cd1a01fef None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.101438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12f674-e107-4a7c-865d-019cd1a01fef None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.102345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12f674-e107-4a7c-865d-019cd1a01fef None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-b0e10aff-aacd-4f86-af33-e4e5bdbc2886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:11.103004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:11.103882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:11.105454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:11.105454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:11.105454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:11.105629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12f674-e107-4a7c-865d-019cd1a01fef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:11.105629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12f674-e107-4a7c-865d-019cd1a01fef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.105753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 334/644] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:11.105753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10aff-aacd-4f86-af33-e4e5bdbc2886 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.109415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12f674-e107-4a7c-865d-019cd1a01fef None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:11.132779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 311/645] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Aug 8 16:41:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:11.134215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:11.134330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12f674-e107-4a7c-865d-019cd1a01fef None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.209996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10aff-aacd-4f86-af33-e4e5bdbc2886 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.239186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12f674-e107-4a7c-865d-019cd1a01fef None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.244341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb6385-446d-48a6-8acb-988b538790fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.244939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb6385-446d-48a6-8acb-988b538790fe None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.245211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb6385-446d-48a6-8acb-988b538790fe None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.245476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb6385-446d-48a6-8acb-988b538790fe None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.246134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb6385-446d-48a6-8acb-988b538790fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:11.246785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb6385-446d-48a6-8acb-988b538790fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.247848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb6385-446d-48a6-8acb-988b538790fe None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:11.266906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 335/646] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:11.266906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb6385-446d-48a6-8acb-988b538790fe None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.280422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c12c0-d5fa-49f1-9c72-37f9563a9cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.281054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c12c0-d5fa-49f1-9c72-37f9563a9cbb None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.281330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c12c0-d5fa-49f1-9c72-37f9563a9cbb None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.281601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c12c0-d5fa-49f1-9c72-37f9563a9cbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c12c0-d5fa-49f1-9c72-37f9563a9cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-995c12c0-d5fa-49f1-9c72-37f9563a9cbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:11.302953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:11.303990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:11.305083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 312/647] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:11.306095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c12c0-d5fa-49f1-9c72-37f9563a9cbb None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.312419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:11.367216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:11.380140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb6385-446d-48a6-8acb-988b538790fe None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.407945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c12c0-d5fa-49f1-9c72-37f9563a9cbb None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.415512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cf1ea-9561-4b82-aa00-3816097bd8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.416119 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cf1ea-9561-4b82-aa00-3816097bd8f0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.416396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cf1ea-9561-4b82-aa00-3816097bd8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.416666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cf1ea-9561-4b82-aa00-3816097bd8f0 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.417311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cf1ea-9561-4b82-aa00-3816097bd8f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:11.417611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cf1ea-9561-4b82-aa00-3816097bd8f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.419007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cf1ea-9561-4b82-aa00-3816097bd8f0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:11.430456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 336/648] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:11.431239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cf1ea-9561-4b82-aa00-3816097bd8f0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.447345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e6631-e495-4282-9f00-96245d9b2566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.448907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e6631-e495-4282-9f00-96245d9b2566 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.449189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e6631-e495-4282-9f00-96245d9b2566 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.449458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e6631-e495-4282-9f00-96245d9b2566 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.531881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e6631-e495-4282-9f00-96245d9b2566 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.533364 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e6631-e495-4282-9f00-96245d9b2566 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:11.543137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cf1ea-9561-4b82-aa00-3816097bd8f0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.545271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:11.579348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bceac70-aaa2-447d-9447-4403b93649ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.579899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bceac70-aaa2-447d-9447-4403b93649ee None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.580178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bceac70-aaa2-447d-9447-4403b93649ee None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.580444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bceac70-aaa2-447d-9447-4403b93649ee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.595131 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bceac70-aaa2-447d-9447-4403b93649ee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.596582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bceac70-aaa2-447d-9447-4403b93649ee None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-0bceac70-aaa2-447d-9447-4403b93649ee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:11.608471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:11.609421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 337/649] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:11.610393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bceac70-aaa2-447d-9447-4403b93649ee None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.624835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:11.651835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 313/650] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:41:11.651835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e6631-e495-4282-9f00-96245d9b2566 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.720417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bceac70-aaa2-447d-9447-4403b93649ee None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.754076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe35a7-1f51-4367-a52a-de42dc322199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.754492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e6631-e495-4282-9f00-96245d9b2566 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.754764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe35a7-1f51-4367-a52a-de42dc322199 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.755045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe35a7-1f51-4367-a52a-de42dc322199 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.755309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe35a7-1f51-4367-a52a-de42dc322199 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.755960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe35a7-1f51-4367-a52a-de42dc322199 None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:11.756259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe35a7-1f51-4367-a52a-de42dc322199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.757609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe35a7-1f51-4367-a52a-de42dc322199 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:11.763620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:11.770424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 338/651] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:11.770424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe35a7-1f51-4367-a52a-de42dc322199 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.798585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada7f27-f6b8-4fec-a81d-8ee7b4fc5099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.799372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada7f27-f6b8-4fec-a81d-8ee7b4fc5099 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.799678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada7f27-f6b8-4fec-a81d-8ee7b4fc5099 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.799954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada7f27-f6b8-4fec-a81d-8ee7b4fc5099 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.806665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada7f27-f6b8-4fec-a81d-8ee7b4fc5099 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.808096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada7f27-f6b8-4fec-a81d-8ee7b4fc5099 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-1ada7f27-f6b8-4fec-a81d-8ee7b4fc5099 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:11.819052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:11.819945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:11.820840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:11.820840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:11.820840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:11.820840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:11.820840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:11.820840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:11.820840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:11.820840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:11.820840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 314/652] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:11.821322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada7f27-f6b8-4fec-a81d-8ee7b4fc5099 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.847227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:11.879975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe35a7-1f51-4367-a52a-de42dc322199 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.918424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c381f-3f0f-4068-92ad-5ce174d7581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.919116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c381f-3f0f-4068-92ad-5ce174d7581f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.919548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c381f-3f0f-4068-92ad-5ce174d7581f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.919896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c381f-3f0f-4068-92ad-5ce174d7581f None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.920776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c381f-3f0f-4068-92ad-5ce174d7581f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:11.921150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c381f-3f0f-4068-92ad-5ce174d7581f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.922780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c381f-3f0f-4068-92ad-5ce174d7581f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:11.928895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada7f27-f6b8-4fec-a81d-8ee7b4fc5099 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:11.945879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 339/653] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:11.945879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c381f-3f0f-4068-92ad-5ce174d7581f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:11.968799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0847b9-e09a-402e-934a-79206f5c7326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:11.969506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0847b9-e09a-402e-934a-79206f5c7326 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:11.969881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0847b9-e09a-402e-934a-79206f5c7326 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:11.970223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0847b9-e09a-402e-934a-79206f5c7326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:11.982616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0847b9-e09a-402e-934a-79206f5c7326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:11.984060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0847b9-e09a-402e-934a-79206f5c7326 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-bd0847b9-e09a-402e-934a-79206f5c7326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:12.000457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:12.001613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:12.002747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:12.002747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:12.002747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:12.006378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 315/654] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:12.007113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0847b9-e09a-402e-934a-79206f5c7326 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.029707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:12.052107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c381f-3f0f-4068-92ad-5ce174d7581f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.080046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:12.088475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362512bd-a23e-4c65-b687-4daadc336b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.089132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362512bd-a23e-4c65-b687-4daadc336b61 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.089392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362512bd-a23e-4c65-b687-4daadc336b61 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.089662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362512bd-a23e-4c65-b687-4daadc336b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.099821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362512bd-a23e-4c65-b687-4daadc336b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:12.101259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362512bd-a23e-4c65-b687-4daadc336b61 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.116500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0847b9-e09a-402e-934a-79206f5c7326 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-362512bd-a23e-4c65-b687-4daadc336b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:12.118460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:12.119365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:12.120239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:12.120239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:12.120239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:12.120239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 340/655] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:12.120856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362512bd-a23e-4c65-b687-4daadc336b61 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.155625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17301b8b-696f-444b-9c15-864cd9f3d0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.156240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17301b8b-696f-444b-9c15-864cd9f3d0ac None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.156504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17301b8b-696f-444b-9c15-864cd9f3d0ac None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.156769 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17301b8b-696f-444b-9c15-864cd9f3d0ac None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.157449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17301b8b-696f-444b-9c15-864cd9f3d0ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:12.157750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17301b8b-696f-444b-9c15-864cd9f3d0ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:12.159197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17301b8b-696f-444b-9c15-864cd9f3d0ac None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.176450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 316/656] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:12.176450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17301b8b-696f-444b-9c15-864cd9f3d0ac None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.232209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362512bd-a23e-4c65-b687-4daadc336b61 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.258332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:12.268285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df0665-159f-4e54-bda2-a00eeb5a391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.268958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df0665-159f-4e54-bda2-a00eeb5a391f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.269237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df0665-159f-4e54-bda2-a00eeb5a391f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.276074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df0665-159f-4e54-bda2-a00eeb5a391f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.278724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17301b8b-696f-444b-9c15-864cd9f3d0ac None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.319815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:12.320392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630485d9-d30c-4cbc-91cc-460c9b27715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.321053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630485d9-d30c-4cbc-91cc-460c9b27715d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.321181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630485d9-d30c-4cbc-91cc-460c9b27715d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.321383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630485d9-d30c-4cbc-91cc-460c9b27715d None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.322046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630485d9-d30c-4cbc-91cc-460c9b27715d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:12.322351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630485d9-d30c-4cbc-91cc-460c9b27715d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:12.324178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630485d9-d30c-4cbc-91cc-460c9b27715d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.335840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 317/657] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:12.337395 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630485d9-d30c-4cbc-91cc-460c9b27715d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.340879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df0665-159f-4e54-bda2-a00eeb5a391f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:12.342747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df0665-159f-4e54-bda2-a00eeb5a391f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.443360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630485d9-d30c-4cbc-91cc-460c9b27715d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.445151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 341/658] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:41:12.446701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df0665-159f-4e54-bda2-a00eeb5a391f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.478372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95043c85-89d0-4f4f-a733-12bcfb2a90ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.479005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95043c85-89d0-4f4f-a733-12bcfb2a90ae None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.479283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95043c85-89d0-4f4f-a733-12bcfb2a90ae None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.479577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95043c85-89d0-4f4f-a733-12bcfb2a90ae None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.480216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95043c85-89d0-4f4f-a733-12bcfb2a90ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:12.480517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95043c85-89d0-4f4f-a733-12bcfb2a90ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:12.481896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95043c85-89d0-4f4f-a733-12bcfb2a90ae None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.490411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:12.493570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 318/659] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:12.494384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95043c85-89d0-4f4f-a733-12bcfb2a90ae None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.533045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:12.553703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df0665-159f-4e54-bda2-a00eeb5a391f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.590489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44489302-21ab-4441-85cf-9a00359ff283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.591092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44489302-21ab-4441-85cf-9a00359ff283 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.591366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44489302-21ab-4441-85cf-9a00359ff283 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.591632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44489302-21ab-4441-85cf-9a00359ff283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.592345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44489302-21ab-4441-85cf-9a00359ff283 None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=73bfe655c6d34918b21acb8748b9156d) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:12.592707 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44489302-21ab-4441-85cf-9a00359ff283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:12.594160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44489302-21ab-4441-85cf-9a00359ff283 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.596998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95043c85-89d0-4f4f-a733-12bcfb2a90ae None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.599937 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 342/660] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/role_assignments?user.id=73bfe655c6d34918b21acb8748b9156d&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:12.601074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57728ea-8465-4d9a-858d-9bddb9744363 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.601463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57728ea-8465-4d9a-858d-9bddb9744363 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.601589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57728ea-8465-4d9a-858d-9bddb9744363 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.603489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 343/661] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:12.604050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57728ea-8465-4d9a-858d-9bddb9744363 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.639338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89830d32-c168-4729-99cf-4f64521657ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.639974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89830d32-c168-4729-99cf-4f64521657ee None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.640301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89830d32-c168-4729-99cf-4f64521657ee None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.640582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89830d32-c168-4729-99cf-4f64521657ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.648157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89830d32-c168-4729-99cf-4f64521657ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:12.650264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89830d32-c168-4729-99cf-4f64521657ee None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-89830d32-c168-4729-99cf-4f64521657ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:12.662824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:12.663914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:12.664976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:12.664976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:12.664976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:12.664976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:12.664976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:12.664976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:12.664976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:12.664976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:12.669028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 319/662] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:12.670031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89830d32-c168-4729-99cf-4f64521657ee None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.719271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57728ea-8465-4d9a-858d-9bddb9744363 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.737108 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:12.754578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86d4fe-1b72-45a0-8343-adbf17dcbf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.755205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86d4fe-1b72-45a0-8343-adbf17dcbf8f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.755478 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86d4fe-1b72-45a0-8343-adbf17dcbf8f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.755796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86d4fe-1b72-45a0-8343-adbf17dcbf8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.766859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86d4fe-1b72-45a0-8343-adbf17dcbf8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:12.768359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86d4fe-1b72-45a0-8343-adbf17dcbf8f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.780421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89830d32-c168-4729-99cf-4f64521657ee None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-5c86d4fe-1b72-45a0-8343-adbf17dcbf8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:12.782986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:12.783916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:12.784911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:12.784911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:12.784911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:12.784911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 344/663] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:12.785529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e02b4-337e-4dac-9b16-7f765a4c7ae0 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.785633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e02b4-337e-4dac-9b16-7f765a4c7ae0 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.785861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e02b4-337e-4dac-9b16-7f765a4c7ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.806237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ee7e02b4-337e-4dac-9b16-7f765a4c7ae0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:12.807417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ee7e02b4-337e-4dac-9b16-7f765a4c7ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:12.818819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515c1d9-68b3-4722-a470-9d8e14aaaf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.819453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515c1d9-68b3-4722-a470-9d8e14aaaf76 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.819731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515c1d9-68b3-4722-a470-9d8e14aaaf76 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.820000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515c1d9-68b3-4722-a470-9d8e14aaaf76 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.827614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515c1d9-68b3-4722-a470-9d8e14aaaf76 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:12.829090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515c1d9-68b3-4722-a470-9d8e14aaaf76 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-7515c1d9-68b3-4722-a470-9d8e14aaaf76 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:12.842133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:12.843086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 320/664] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:12.844711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515c1d9-68b3-4722-a470-9d8e14aaaf76 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.858981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:12.889855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 345/665] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:12.889855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e02b4-337e-4dac-9b16-7f765a4c7ae0 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:12.958236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515c1d9-68b3-4722-a470-9d8e14aaaf76 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.982166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:12.993586 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e02b4-337e-4dac-9b16-7f765a4c7ae0 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:12.998021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacdfb6-422b-40f4-a356-71a6018d0580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:12.998021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacdfb6-422b-40f4-a356-71a6018d0580 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:12.998021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacdfb6-422b-40f4-a356-71a6018d0580 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:12.998286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacdfb6-422b-40f4-a356-71a6018d0580 None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:12.998920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacdfb6-422b-40f4-a356-71a6018d0580 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:12.999220 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacdfb6-422b-40f4-a356-71a6018d0580 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:13.000679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacdfb6-422b-40f4-a356-71a6018d0580 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:13.017556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 321/666] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:13.017556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacdfb6-422b-40f4-a356-71a6018d0580 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.025355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:13.034075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97878d41-e269-44f2-a363-f9996df72c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:13.040298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97878d41-e269-44f2-a363-f9996df72c33 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:13.040298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97878d41-e269-44f2-a363-f9996df72c33 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:13.040298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97878d41-e269-44f2-a363-f9996df72c33 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:13.040298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97878d41-e269-44f2-a363-f9996df72c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:13.040298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97878d41-e269-44f2-a363-f9996df72c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:13.040298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97878d41-e269-44f2-a363-f9996df72c33 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:13.055258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 346/667] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:13.056129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97878d41-e269-44f2-a363-f9996df72c33 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.127410 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacdfb6-422b-40f4-a356-71a6018d0580 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:13.162570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367cac2-53e8-46f2-a75f-395ff0befc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:13.163490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367cac2-53e8-46f2-a75f-395ff0befc26 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:13.163774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367cac2-53e8-46f2-a75f-395ff0befc26 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:13.164761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367cac2-53e8-46f2-a75f-395ff0befc26 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:13.165500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367cac2-53e8-46f2-a75f-395ff0befc26 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:13.165869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367cac2-53e8-46f2-a75f-395ff0befc26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:13.166833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97878d41-e269-44f2-a363-f9996df72c33 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:13.167277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367cac2-53e8-46f2-a75f-395ff0befc26 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:13.189149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 322/668] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:13.189149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367cac2-53e8-46f2-a75f-395ff0befc26 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.204954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a147e6f-2aa4-46da-a732-82021a935bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:13.205577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a147e6f-2aa4-46da-a732-82021a935bc9 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:13.205876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a147e6f-2aa4-46da-a732-82021a935bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:13.206144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a147e6f-2aa4-46da-a732-82021a935bc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:13.217344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a147e6f-2aa4-46da-a732-82021a935bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-3a147e6f-2aa4-46da-a732-82021a935bc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:13.222548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:13.223496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:13.224323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 347/669] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:13.225549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a147e6f-2aa4-46da-a732-82021a935bc9 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.235248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:13.274986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:13.298760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367cac2-53e8-46f2-a75f-395ff0befc26 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:13.334322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac93e98-8265-49e5-a37d-ef246b7316d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:13.334922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac93e98-8265-49e5-a37d-ef246b7316d9 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:13.335190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac93e98-8265-49e5-a37d-ef246b7316d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:13.335451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac93e98-8265-49e5-a37d-ef246b7316d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:13.336767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac93e98-8265-49e5-a37d-ef246b7316d9 None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=54b05cbc41b844d8932ba4c15d960289) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:13.337069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac93e98-8265-49e5-a37d-ef246b7316d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:13.338498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac93e98-8265-49e5-a37d-ef246b7316d9 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:13.340026 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a147e6f-2aa4-46da-a732-82021a935bc9 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:13.353820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 323/670] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/role_assignments?user.id=54b05cbc41b844d8932ba4c15d960289&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:13.353820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac93e98-8265-49e5-a37d-ef246b7316d9 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.397803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76434a74-bcf3-4cfe-a1a3-b76728e5dc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:13.399157 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76434a74-bcf3-4cfe-a1a3-b76728e5dc19 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:13.399434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76434a74-bcf3-4cfe-a1a3-b76728e5dc19 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:13.399698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76434a74-bcf3-4cfe-a1a3-b76728e5dc19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:13.413744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76434a74-bcf3-4cfe-a1a3-b76728e5dc19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:13.417010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76434a74-bcf3-4cfe-a1a3-b76728e5dc19 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-76434a74-bcf3-4cfe-a1a3-b76728e5dc19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:13.434414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:13.435357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:13.436217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:13.436217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:13.436217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:13.436217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:13.436217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:13.436217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:13.436217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:13.436217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:13.443892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 348/671] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:13.443892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76434a74-bcf3-4cfe-a1a3-b76728e5dc19 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.495022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac93e98-8265-49e5-a37d-ef246b7316d9 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:13.497693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:13.548210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3de44-df04-4942-92b4-d332124ee399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:13.549132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3de44-df04-4942-92b4-d332124ee399 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:13.549419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3de44-df04-4942-92b4-d332124ee399 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:13.549690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3de44-df04-4942-92b4-d332124ee399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:13.579509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76434a74-bcf3-4cfe-a1a3-b76728e5dc19 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:13.597516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:13.633003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cb2ff-d457-4078-bbab-1414baf3d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:13.634313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cb2ff-d457-4078-bbab-1414baf3d83c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:13.634473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3de44-df04-4942-92b4-d332124ee399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:13.634939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cb2ff-d457-4078-bbab-1414baf3d83c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:13.635393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cb2ff-d457-4078-bbab-1414baf3d83c None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:13.635941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3de44-df04-4942-92b4-d332124ee399 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:13.636420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cb2ff-d457-4078-bbab-1414baf3d83c None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:13.636725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cb2ff-d457-4078-bbab-1414baf3d83c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:13.638832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cb2ff-d457-4078-bbab-1414baf3d83c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:13.652633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 349/672] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:13.652633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cb2ff-d457-4078-bbab-1414baf3d83c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.753932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 324/673] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:41:13.753932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3de44-df04-4942-92b4-d332124ee399 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.756444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:13.788534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cb2ff-d457-4078-bbab-1414baf3d83c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:13.831198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c13203-feb1-4fd8-8c25-48483fc971a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:13.832139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c13203-feb1-4fd8-8c25-48483fc971a6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:13.832608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c13203-feb1-4fd8-8c25-48483fc971a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:13.833056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c13203-feb1-4fd8-8c25-48483fc971a6 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:13.833962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c13203-feb1-4fd8-8c25-48483fc971a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:13.834456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c13203-feb1-4fd8-8c25-48483fc971a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:13.836680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c13203-feb1-4fd8-8c25-48483fc971a6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:13.850557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 350/674] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:13.852359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c13203-feb1-4fd8-8c25-48483fc971a6 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.863445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3de44-df04-4942-92b4-d332124ee399 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:13.882427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:13.899251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de40866-edde-4bf6-a2b1-9db35002277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:13.899970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de40866-edde-4bf6-a2b1-9db35002277e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:13.900320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de40866-edde-4bf6-a2b1-9db35002277e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:13.900663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de40866-edde-4bf6-a2b1-9db35002277e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:13.911653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de40866-edde-4bf6-a2b1-9db35002277e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:13.913212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de40866-edde-4bf6-a2b1-9db35002277e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-0de40866-edde-4bf6-a2b1-9db35002277e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:13.928621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:13.929573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:13.931354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:13.931354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:13.931354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:13.935107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 325/675] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:13.936056 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de40866-edde-4bf6-a2b1-9db35002277e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:13.966342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c13203-feb1-4fd8-8c25-48483fc971a6 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:13.994300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:14.003356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63011f-cf8c-471f-a652-92e34241d503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:14.006876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63011f-cf8c-471f-a652-92e34241d503 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.006876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63011f-cf8c-471f-a652-92e34241d503 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.006876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63011f-cf8c-471f-a652-92e34241d503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.018500 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63011f-cf8c-471f-a652-92e34241d503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:14.020246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63011f-cf8c-471f-a652-92e34241d503 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-6b63011f-cf8c-471f-a652-92e34241d503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:14.041128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:14.042159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:14.043024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:14.043024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:14.043024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:14.045858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 351/676] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:14.046853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63011f-cf8c-471f-a652-92e34241d503 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:14.049355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de40866-edde-4bf6-a2b1-9db35002277e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:14.088646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c913f07-30af-4f2e-a90b-0a3d3f0c6891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:14.089633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c913f07-30af-4f2e-a90b-0a3d3f0c6891 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.090234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c913f07-30af-4f2e-a90b-0a3d3f0c6891 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.090721 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c913f07-30af-4f2e-a90b-0a3d3f0c6891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.091689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c913f07-30af-4f2e-a90b-0a3d3f0c6891 None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=e860aad24688491c9a89ddfb35922eb4) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:14.092201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c913f07-30af-4f2e-a90b-0a3d3f0c6891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:14.093926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c913f07-30af-4f2e-a90b-0a3d3f0c6891 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:14.100799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 326/677] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:13 2022] GET /identity/v3/role_assignments?user.id=e860aad24688491c9a89ddfb35922eb4&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:14.102529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c913f07-30af-4f2e-a90b-0a3d3f0c6891 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:14.151351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:14.171548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63011f-cf8c-471f-a652-92e34241d503 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:14.215892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebf089-6b6c-432c-8e4a-2ad271784d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:14.217240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebf089-6b6c-432c-8e4a-2ad271784d39 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.217728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebf089-6b6c-432c-8e4a-2ad271784d39 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.218342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebf089-6b6c-432c-8e4a-2ad271784d39 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.219501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebf089-6b6c-432c-8e4a-2ad271784d39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:14.220007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebf089-6b6c-432c-8e4a-2ad271784d39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:14.221605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebf089-6b6c-432c-8e4a-2ad271784d39 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:14.242090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 352/678] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:14.247413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebf089-6b6c-432c-8e4a-2ad271784d39 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:14.249743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c913f07-30af-4f2e-a90b-0a3d3f0c6891 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:14.297587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:14.330800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241163fd-7db0-417e-949a-5373a3c419fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:14.331421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241163fd-7db0-417e-949a-5373a3c419fe None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.331700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241163fd-7db0-417e-949a-5373a3c419fe None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.331972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241163fd-7db0-417e-949a-5373a3c419fe None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.332714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241163fd-7db0-417e-949a-5373a3c419fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:14.333021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241163fd-7db0-417e-949a-5373a3c419fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:14.335290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241163fd-7db0-417e-949a-5373a3c419fe None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:14.361936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 327/679] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:14.369443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241163fd-7db0-417e-949a-5373a3c419fe None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:14.416854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebf089-6b6c-432c-8e4a-2ad271784d39 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:14.456119 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303aa9ae-2748-4fe0-a1f7-b395f8770316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:14.456757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303aa9ae-2748-4fe0-a1f7-b395f8770316 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.457025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303aa9ae-2748-4fe0-a1f7-b395f8770316 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.457337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303aa9ae-2748-4fe0-a1f7-b395f8770316 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.466477 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303aa9ae-2748-4fe0-a1f7-b395f8770316 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:14.468111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303aa9ae-2748-4fe0-a1f7-b395f8770316 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:14.471322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-303aa9ae-2748-4fe0-a1f7-b395f8770316 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:14.480717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:14.481694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:14.482661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 353/680] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:14.489909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303aa9ae-2748-4fe0-a1f7-b395f8770316 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:14.494609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241163fd-7db0-417e-949a-5373a3c419fe None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:14.542161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff74e4-e3d0-410f-99f6-2a5a320b65e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:14.542910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff74e4-e3d0-410f-99f6-2a5a320b65e3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.543608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff74e4-e3d0-410f-99f6-2a5a320b65e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.543883 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff74e4-e3d0-410f-99f6-2a5a320b65e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.553618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff74e4-e3d0-410f-99f6-2a5a320b65e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:14.555851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff74e4-e3d0-410f-99f6-2a5a320b65e3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-08ff74e4-e3d0-410f-99f6-2a5a320b65e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:14.570996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:14.573122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:14.574344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:14.574344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:14.574344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:14.574344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:14.574344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:14.574344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:14.574344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:14.574344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:14.574344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 328/681] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:14.578882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff74e4-e3d0-410f-99f6-2a5a320b65e3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:14.629595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:14.695794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303aa9ae-2748-4fe0-a1f7-b395f8770316 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:14.699741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff74e4-e3d0-410f-99f6-2a5a320b65e3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:14.740372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ae6c6-6799-474e-bd00-e7a6bd5aa076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:14.741349 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ae6c6-6799-474e-bd00-e7a6bd5aa076 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.741859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ae6c6-6799-474e-bd00-e7a6bd5aa076 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.742391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ae6c6-6799-474e-bd00-e7a6bd5aa076 None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.743494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ae6c6-6799-474e-bd00-e7a6bd5aa076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:14.744076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ae6c6-6799-474e-bd00-e7a6bd5aa076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:14.745704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ae6c6-6799-474e-bd00-e7a6bd5aa076 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:14.759809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 329/682] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:14.765274 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ae6c6-6799-474e-bd00-e7a6bd5aa076 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:14.783475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f46ea4-0258-464d-8958-502509bd1ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:14.784488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46ea4-0258-464d-8958-502509bd1ff6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.784998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46ea4-0258-464d-8958-502509bd1ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.785527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46ea4-0258-464d-8958-502509bd1ff6 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.788788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46ea4-0258-464d-8958-502509bd1ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:14.789965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46ea4-0258-464d-8958-502509bd1ff6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:14.791646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46ea4-0258-464d-8958-502509bd1ff6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:14.827190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 354/683] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Aug 8 16:41:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:14.832382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f46ea4-0258-464d-8958-502509bd1ff6 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:14.916528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:14.923178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:14.927567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ae6c6-6799-474e-bd00-e7a6bd5aa076 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:14.967904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f46ea4-0258-464d-8958-502509bd1ff6 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:14.972496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7475e25-5d34-46f5-a882-0de500c21efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:14.973178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7475e25-5d34-46f5-a882-0de500c21efb None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.973524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7475e25-5d34-46f5-a882-0de500c21efb None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.973890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7475e25-5d34-46f5-a882-0de500c21efb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.974685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7475e25-5d34-46f5-a882-0de500c21efb None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=f50ee57e2418417cbe68915ca3f00f76) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:14.975062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7475e25-5d34-46f5-a882-0de500c21efb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:14.977097 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7475e25-5d34-46f5-a882-0de500c21efb None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:14.983958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 330/684] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:14 2022] GET /identity/v3/role_assignments?user.id=f50ee57e2418417cbe68915ca3f00f76&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:14.985501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0d35f-f22d-4dac-bf03-9db1b6bbe531 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:14.994497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0d35f-f22d-4dac-bf03-9db1b6bbe531 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:14.994944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0d35f-f22d-4dac-bf03-9db1b6bbe531 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:14.996601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 331/685] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:15.002734 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ef908-e739-4ea9-90e0-cfedfa1bcfde None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:15.003142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ef908-e739-4ea9-90e0-cfedfa1bcfde None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:15.003521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ef908-e739-4ea9-90e0-cfedfa1bcfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:15.043159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ce7ef908-e739-4ea9-90e0-cfedfa1bcfde None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:15.044683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ce7ef908-e739-4ea9-90e0-cfedfa1bcfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:15.145205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 332/686] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:41:15.171803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc34ac-aa9d-4111-bc7b-6723415fd035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:15.173048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc34ac-aa9d-4111-bc7b-6723415fd035 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:15.173468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc34ac-aa9d-4111-bc7b-6723415fd035 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:15.173934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc34ac-aa9d-4111-bc7b-6723415fd035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:15.176903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ef908-e739-4ea9-90e0-cfedfa1bcfde None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:15.252917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc34ac-aa9d-4111-bc7b-6723415fd035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:15.254577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc34ac-aa9d-4111-bc7b-6723415fd035 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:15.303169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:15.312158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ef908-e739-4ea9-90e0-cfedfa1bcfde None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:15.323932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:15.371430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a65b60-93a2-47c1-8fe2-cc8f53a67391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:15.372066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a65b60-93a2-47c1-8fe2-cc8f53a67391 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:15.372341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a65b60-93a2-47c1-8fe2-cc8f53a67391 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:15.372609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a65b60-93a2-47c1-8fe2-cc8f53a67391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:15.387520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a65b60-93a2-47c1-8fe2-cc8f53a67391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:15.388966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a65b60-93a2-47c1-8fe2-cc8f53a67391 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:15.391801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 355/687] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:41:15.398118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc34ac-aa9d-4111-bc7b-6723415fd035 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a6a65b60-93a2-47c1-8fe2-cc8f53a67391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:15.422156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:15.423165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:15.424050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:15.424050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:15.424050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:15.429131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 333/688] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:15.432816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a65b60-93a2-47c1-8fe2-cc8f53a67391 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:15.556907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc34ac-aa9d-4111-bc7b-6723415fd035 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:15.608948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50f1f7-8bc2-4875-acee-3a6816673bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:15.609901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50f1f7-8bc2-4875-acee-3a6816673bbd None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:15.610328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50f1f7-8bc2-4875-acee-3a6816673bbd None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:15.611265 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50f1f7-8bc2-4875-acee-3a6816673bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:15.622694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50f1f7-8bc2-4875-acee-3a6816673bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:15.624597 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50f1f7-8bc2-4875-acee-3a6816673bbd None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-2e50f1f7-8bc2-4875-acee-3a6816673bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:15.640918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:15.642118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:15.643243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:15.643243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:15.643243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:15.645404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 356/689] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:15.646562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d982c-8eb7-4bd3-93cb-338db3e354ac None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:15.646840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d982c-8eb7-4bd3-93cb-338db3e354ac None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:15.647108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d982c-8eb7-4bd3-93cb-338db3e354ac None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:15.648607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 357/690] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:15.651475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d982c-8eb7-4bd3-93cb-338db3e354ac None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:15.657268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a65b60-93a2-47c1-8fe2-cc8f53a67391 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:15.670097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:15.708640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0101684-0659-4f99-bdd1-98e3d38df544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:15.709371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0101684-0659-4f99-bdd1-98e3d38df544 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:15.709725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0101684-0659-4f99-bdd1-98e3d38df544 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:15.710099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0101684-0659-4f99-bdd1-98e3d38df544 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:15.710860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0101684-0659-4f99-bdd1-98e3d38df544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:15.711754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0101684-0659-4f99-bdd1-98e3d38df544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:15.714501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0101684-0659-4f99-bdd1-98e3d38df544 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:15.736431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 334/691] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:15.736431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fef2b-b9f5-4158-a47e-b616c05adcf9 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:15.736431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fef2b-b9f5-4158-a47e-b616c05adcf9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:15.736431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fef2b-b9f5-4158-a47e-b616c05adcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:15.743329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:15.760171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-dc2fef2b-b9f5-4158-a47e-b616c05adcf9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:15.761763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-dc2fef2b-b9f5-4158-a47e-b616c05adcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:15.787826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d982c-8eb7-4bd3-93cb-338db3e354ac None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:15.833944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 335/692] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:15.835231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fef2b-b9f5-4158-a47e-b616c05adcf9 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:15.835395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f2893-6b39-45ab-9dc0-a03b28225021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:15.836158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f2893-6b39-45ab-9dc0-a03b28225021 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:15.836506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f2893-6b39-45ab-9dc0-a03b28225021 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:15.836846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f2893-6b39-45ab-9dc0-a03b28225021 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:15.837561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f2893-6b39-45ab-9dc0-a03b28225021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:15.837958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f2893-6b39-45ab-9dc0-a03b28225021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:15.840616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f2893-6b39-45ab-9dc0-a03b28225021 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:15.856229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 358/693] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:15.862014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f2893-6b39-45ab-9dc0-a03b28225021 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:15.945731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fef2b-b9f5-4158-a47e-b616c05adcf9 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:15.955991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:15.982725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfeae2-7a86-4ee9-84aa-70e31afd16b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:15.983638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfeae2-7a86-4ee9-84aa-70e31afd16b7 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:15.984124 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfeae2-7a86-4ee9-84aa-70e31afd16b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:15.984575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfeae2-7a86-4ee9-84aa-70e31afd16b7 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:15.985382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfeae2-7a86-4ee9-84aa-70e31afd16b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:15.987089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfeae2-7a86-4ee9-84aa-70e31afd16b7 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:15.993946 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f2893-6b39-45ab-9dc0-a03b28225021 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:15.995682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:16.030413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1820cc0-6d9f-414b-8598-512686605c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:16.031045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1820cc0-6d9f-414b-8598-512686605c8f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:16.031324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1820cc0-6d9f-414b-8598-512686605c8f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:16.031595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1820cc0-6d9f-414b-8598-512686605c8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:16.038365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1820cc0-6d9f-414b-8598-512686605c8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:16.040355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1820cc0-6d9f-414b-8598-512686605c8f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-e1820cc0-6d9f-414b-8598-512686605c8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:16.051331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:16.052219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:16.053095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:16.053095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:16.053095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:16.053095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:16.053095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:16.053095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:16.053095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:16.053095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:16.057063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 359/694] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:16.057819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1820cc0-6d9f-414b-8598-512686605c8f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:16.069971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 336/695] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:41:15 2022] POST /identity/v3/users => generated 254 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:16.070937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfeae2-7a86-4ee9-84aa-70e31afd16b7 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:16.184284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1820cc0-6d9f-414b-8598-512686605c8f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:16.228847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:16.238333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe2fb7-5369-4ccf-906f-9ffb4a248f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:16.242614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe2fb7-5369-4ccf-906f-9ffb4a248f86 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:16.242614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe2fb7-5369-4ccf-906f-9ffb4a248f86 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:16.242614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe2fb7-5369-4ccf-906f-9ffb4a248f86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:16.269641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe2fb7-5369-4ccf-906f-9ffb4a248f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-bdfe2fb7-5369-4ccf-906f-9ffb4a248f86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:16.281965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:16.282943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:16.283792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:16.285990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 360/696] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:16.291789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe2fb7-5369-4ccf-906f-9ffb4a248f86 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:16.319185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfeae2-7a86-4ee9-84aa-70e31afd16b7 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:16.361023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0c19c-1b5c-436d-b8ae-a51ee284f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:16.362426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0c19c-1b5c-436d-b8ae-a51ee284f119 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:16.363157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0c19c-1b5c-436d-b8ae-a51ee284f119 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:16.363598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0c19c-1b5c-436d-b8ae-a51ee284f119 None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:16.364753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0c19c-1b5c-436d-b8ae-a51ee284f119 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:16.365275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0c19c-1b5c-436d-b8ae-a51ee284f119 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:16.367659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0c19c-1b5c-436d-b8ae-a51ee284f119 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:16.385779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 337/697] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:16.391353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0c19c-1b5c-436d-b8ae-a51ee284f119 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:16.421949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe2fb7-5369-4ccf-906f-9ffb4a248f86 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:16.464826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:16.465858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24a886-f188-4e86-87b6-e850ceaffe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:16.466585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24a886-f188-4e86-87b6-e850ceaffe91 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:16.466953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24a886-f188-4e86-87b6-e850ceaffe91 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:16.467318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24a886-f188-4e86-87b6-e850ceaffe91 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:16.468139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24a886-f188-4e86-87b6-e850ceaffe91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:16.468539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24a886-f188-4e86-87b6-e850ceaffe91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:16.470032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24a886-f188-4e86-87b6-e850ceaffe91 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:16.481621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 361/698] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:16.490543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24a886-f188-4e86-87b6-e850ceaffe91 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:16.536753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0c19c-1b5c-436d-b8ae-a51ee284f119 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:16.552279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:16.581112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035fa52-3f4b-4a3d-b716-b3f3b916a585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:16.581743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035fa52-3f4b-4a3d-b716-b3f3b916a585 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:16.582045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035fa52-3f4b-4a3d-b716-b3f3b916a585 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:16.582365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035fa52-3f4b-4a3d-b716-b3f3b916a585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:16.584246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035fa52-3f4b-4a3d-b716-b3f3b916a585 None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=92130daa3c3a49c2adc597dc563b2973) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:16.584561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035fa52-3f4b-4a3d-b716-b3f3b916a585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:16.586246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035fa52-3f4b-4a3d-b716-b3f3b916a585 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:16.594646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 338/699] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:16 2022] GET /identity/v3/role_assignments?user.id=92130daa3c3a49c2adc597dc563b2973&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:16.618696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24a886-f188-4e86-87b6-e850ceaffe91 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:16.666610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a092fa1-1b86-475d-82f2-871cec4f42f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:16.667319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a092fa1-1b86-475d-82f2-871cec4f42f8 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:16.667596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a092fa1-1b86-475d-82f2-871cec4f42f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:16.667863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a092fa1-1b86-475d-82f2-871cec4f42f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:16.681034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a092fa1-1b86-475d-82f2-871cec4f42f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:16.686624 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a092fa1-1b86-475d-82f2-871cec4f42f8 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-9a092fa1-1b86-475d-82f2-871cec4f42f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:16.705692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:16.706717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:16.707601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:16.707601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:16.707601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:16.707601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 362/700] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:16.714639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035fa52-3f4b-4a3d-b716-b3f3b916a585 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:16.744904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c8aa8-f003-482d-9f31-49f268568175 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:16.745100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c8aa8-f003-482d-9f31-49f268568175 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:16.745348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c8aa8-f003-482d-9f31-49f268568175 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:16.746978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 363/701] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:16.753031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f3aa8-865c-4b0f-a1ac-98037b7d6233 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:16.753338 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f3aa8-865c-4b0f-a1ac-98037b7d6233 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:16.753573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f3aa8-865c-4b0f-a1ac-98037b7d6233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:16.790947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-035f3aa8-865c-4b0f-a1ac-98037b7d6233 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:16.792176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-035f3aa8-865c-4b0f-a1ac-98037b7d6233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:16.843617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035fa52-3f4b-4a3d-b716-b3f3b916a585 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:16.858558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:16.881376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ac0d1-a4c7-4bbd-9244-78a66474fb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:16.882371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ac0d1-a4c7-4bbd-9244-78a66474fb03 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:16.883456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ac0d1-a4c7-4bbd-9244-78a66474fb03 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:16.883924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ac0d1-a4c7-4bbd-9244-78a66474fb03 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:16.884768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ac0d1-a4c7-4bbd-9244-78a66474fb03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:16.885337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ac0d1-a4c7-4bbd-9244-78a66474fb03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:16.886923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ac0d1-a4c7-4bbd-9244-78a66474fb03 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:16.896135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 364/702] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:41:16.902354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 339/703] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:16.907212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f3aa8-865c-4b0f-a1ac-98037b7d6233 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:16.920786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ac0d1-a4c7-4bbd-9244-78a66474fb03 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:16.949376 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:17.027042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f3aa8-865c-4b0f-a1ac-98037b7d6233 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:17.030736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ac0d1-a4c7-4bbd-9244-78a66474fb03 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:17.065054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29831c7a-3fe2-4336-8869-e53ca462147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:17.065631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa0854-7d4a-4094-84f9-49ebab3a8ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:17.065754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29831c7a-3fe2-4336-8869-e53ca462147a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:17.065995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29831c7a-3fe2-4336-8869-e53ca462147a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:17.066264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29831c7a-3fe2-4336-8869-e53ca462147a None admin] PATH_INFO: `/v3/users/nova` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:17.066363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa0854-7d4a-4094-84f9-49ebab3a8ab5 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:17.066556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa0854-7d4a-4094-84f9-49ebab3a8ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:17.066825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa0854-7d4a-4094-84f9-49ebab3a8ab5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:17.067402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa0854-7d4a-4094-84f9-49ebab3a8ab5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:17.068759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa0854-7d4a-4094-84f9-49ebab3a8ab5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:17.074797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29831c7a-3fe2-4336-8869-e53ca462147a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:17.076229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29831c7a-3fe2-4336-8869-e53ca462147a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-69fa0854-7d4a-4094-84f9-49ebab3a8ab5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:17.080244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:17.081318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:17.082251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 340/704] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Aug 8 16:41:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-29831c7a-3fe2-4336-8869-e53ca462147a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:17.094032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:17.096175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:17.097526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:17.098386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa0854-7d4a-4094-84f9-49ebab3a8ab5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:17.098463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 365/705] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Aug 8 16:41:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:17.105756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29831c7a-3fe2-4336-8869-e53ca462147a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:17.147608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:17.220319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa0854-7d4a-4094-84f9-49ebab3a8ab5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:17.229369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:17.238484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29831c7a-3fe2-4336-8869-e53ca462147a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:17.269014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a866e5b-9462-4cf4-9e51-5b6fe7013854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:17.270227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a866e5b-9462-4cf4-9e51-5b6fe7013854 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:17.270691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a866e5b-9462-4cf4-9e51-5b6fe7013854 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:17.271127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a866e5b-9462-4cf4-9e51-5b6fe7013854 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:17.272016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a866e5b-9462-4cf4-9e51-5b6fe7013854 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:17.272486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a866e5b-9462-4cf4-9e51-5b6fe7013854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:17.274150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a866e5b-9462-4cf4-9e51-5b6fe7013854 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:17.281491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e09ee2-f704-4835-bad4-c2e5e9d9bf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:17.283575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e09ee2-f704-4835-bad4-c2e5e9d9bf52 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:17.285521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 341/706] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Aug 8 16:41:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:17.286141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e09ee2-f704-4835-bad4-c2e5e9d9bf52 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:17.286682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e09ee2-f704-4835-bad4-c2e5e9d9bf52 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:17.287681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e09ee2-f704-4835-bad4-c2e5e9d9bf52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:17.288190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e09ee2-f704-4835-bad4-c2e5e9d9bf52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:17.291373 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a866e5b-9462-4cf4-9e51-5b6fe7013854 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:17.294501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e09ee2-f704-4835-bad4-c2e5e9d9bf52 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:17.331262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 366/707] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Aug 8 16:41:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:17.336941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e09ee2-f704-4835-bad4-c2e5e9d9bf52 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:17.449319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a866e5b-9462-4cf4-9e51-5b6fe7013854 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:17.451475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:17.488434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a5104-6c66-4fb5-8664-8ece369d4fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:17.489401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5104-6c66-4fb5-8664-8ece369d4fcc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:17.489915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5104-6c66-4fb5-8664-8ece369d4fcc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:17.490386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5104-6c66-4fb5-8664-8ece369d4fcc None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:17.491335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a5104-6c66-4fb5-8664-8ece369d4fcc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:17.491878 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a5104-6c66-4fb5-8664-8ece369d4fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:17.493459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a5104-6c66-4fb5-8664-8ece369d4fcc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:17.500307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 342/708] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Aug 8 16:41:17 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:17.526612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e09ee2-f704-4835-bad4-c2e5e9d9bf52 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:17.546913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9370ff0-bb8b-42b2-82be-c9396af18928 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:17.547551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9370ff0-bb8b-42b2-82be-c9396af18928 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:17.547909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9370ff0-bb8b-42b2-82be-c9396af18928 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:17.549737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 343/709] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:41:17.556681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd3cf3-30a6-4ff1-840c-aeff94f2494e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:17.557059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd3cf3-30a6-4ff1-840c-aeff94f2494e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:17.557433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd3cf3-30a6-4ff1-840c-aeff94f2494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:17.567169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7cfeb-877e-4359-a9ba-6a9e4c9cb0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:17.568056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7cfeb-877e-4359-a9ba-6a9e4c9cb0b3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:17.568489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7cfeb-877e-4359-a9ba-6a9e4c9cb0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:17.568916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7cfeb-877e-4359-a9ba-6a9e4c9cb0b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:17.579483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-0afd3cf3-30a6-4ff1-840c-aeff94f2494e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:17.580951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0afd3cf3-30a6-4ff1-840c-aeff94f2494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:17.606019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:17.662644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 344/710] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:17.684215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7cfeb-877e-4359-a9ba-6a9e4c9cb0b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:17.685675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7cfeb-877e-4359-a9ba-6a9e4c9cb0b3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:17.702750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd3cf3-30a6-4ff1-840c-aeff94f2494e None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:17.801988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 367/711] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:41:17.807650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7cfeb-877e-4359-a9ba-6a9e4c9cb0b3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:17.849594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:17.856842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd3cf3-30a6-4ff1-840c-aeff94f2494e None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:17.896687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:17.926498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd5755-7002-4f68-97a5-eac62da88d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:17.927158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd5755-7002-4f68-97a5-eac62da88d80 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:17.927424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd5755-7002-4f68-97a5-eac62da88d80 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:17.927697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd5755-7002-4f68-97a5-eac62da88d80 None admin] PATH_INFO: `/v3/services/network` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:17.928293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd5755-7002-4f68-97a5-eac62da88d80 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:17.930264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd5755-7002-4f68-97a5-eac62da88d80 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-6dfd5755-7002-4f68-97a5-eac62da88d80 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:17.939437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:17.940441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:17.941294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 345/712] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:17 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:17.946422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd5755-7002-4f68-97a5-eac62da88d80 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:17.972209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7cfeb-877e-4359-a9ba-6a9e4c9cb0b3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:18.046485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956ebd5-4ac6-4269-8457-ba7d7391bde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:18.047121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956ebd5-4ac6-4269-8457-ba7d7391bde1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:18.047391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956ebd5-4ac6-4269-8457-ba7d7391bde1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:18.047649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956ebd5-4ac6-4269-8457-ba7d7391bde1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:18.059109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956ebd5-4ac6-4269-8457-ba7d7391bde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:18.060483 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956ebd5-4ac6-4269-8457-ba7d7391bde1 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-5956ebd5-4ac6-4269-8457-ba7d7391bde1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:18.074497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:18.075465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:18.076366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:18.076366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:18.076366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:18.076366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 368/713] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:18.081051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956ebd5-4ac6-4269-8457-ba7d7391bde1 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:18.093067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd5755-7002-4f68-97a5-eac62da88d80 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:18.153248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33074a6-eadc-498b-b0b1-43c6b436d09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:18.153889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33074a6-eadc-498b-b0b1-43c6b436d09c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:18.154174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33074a6-eadc-498b-b0b1-43c6b436d09c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:18.154450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33074a6-eadc-498b-b0b1-43c6b436d09c None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:18.155092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33074a6-eadc-498b-b0b1-43c6b436d09c None admin] RBAC: Adding query filter params (name=network) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:18.155396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33074a6-eadc-498b-b0b1-43c6b436d09c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:18.156767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33074a6-eadc-498b-b0b1-43c6b436d09c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:18.163512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 346/714] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:18.169369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33074a6-eadc-498b-b0b1-43c6b436d09c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:18.198574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956ebd5-4ac6-4269-8457-ba7d7391bde1 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:18.201062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:18.223335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:18.236902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017e0f7-b685-403b-bac7-2c14426f5e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:18.237589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017e0f7-b685-403b-bac7-2c14426f5e47 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:18.238000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017e0f7-b685-403b-bac7-2c14426f5e47 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:18.238123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017e0f7-b685-403b-bac7-2c14426f5e47 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:18.238715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017e0f7-b685-403b-bac7-2c14426f5e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:18.239018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017e0f7-b685-403b-bac7-2c14426f5e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:18.240439 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017e0f7-b685-403b-bac7-2c14426f5e47 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:18.252789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 369/715] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:18.258193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017e0f7-b685-403b-bac7-2c14426f5e47 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:18.469442 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017e0f7-b685-403b-bac7-2c14426f5e47 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:18.478046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33074a6-eadc-498b-b0b1-43c6b436d09c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:18.537418 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034acc-69d6-4c06-b06c-ef043e11f021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:18.538076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034acc-69d6-4c06-b06c-ef043e11f021 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:18.538358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034acc-69d6-4c06-b06c-ef043e11f021 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:18.538681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034acc-69d6-4c06-b06c-ef043e11f021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:18.539881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ecfb8-b7d7-4c9f-8b09-b16f3779b556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:18.540485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ecfb8-b7d7-4c9f-8b09-b16f3779b556 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:18.540754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ecfb8-b7d7-4c9f-8b09-b16f3779b556 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:18.541017 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ecfb8-b7d7-4c9f-8b09-b16f3779b556 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:18.541652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ecfb8-b7d7-4c9f-8b09-b16f3779b556 None admin] RBAC: Adding query filter params (type=network) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:18.541978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ecfb8-b7d7-4c9f-8b09-b16f3779b556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:18.543353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ecfb8-b7d7-4c9f-8b09-b16f3779b556 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:18.552718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd034acc-69d6-4c06-b06c-ef043e11f021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:18.554227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd034acc-69d6-4c06-b06c-ef043e11f021 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:18.558132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 347/716] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:18 2022] GET /identity/v3/services?type=network => generated 124 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-bd034acc-69d6-4c06-b06c-ef043e11f021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:18.568792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:18.569739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:18.571399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:18.571399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:18.571399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:18.571399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:18.571399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:18.571399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:18.571399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:18.571399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:18.571399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 370/717] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:18.578935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ecfb8-b7d7-4c9f-8b09-b16f3779b556 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:18.710028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:18.729591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ecfb8-b7d7-4c9f-8b09-b16f3779b556 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:18.773613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81964dbd-7f42-4e43-8f7b-6316add8da1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:18.774262 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81964dbd-7f42-4e43-8f7b-6316add8da1a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:18.774537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81964dbd-7f42-4e43-8f7b-6316add8da1a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:18.774805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81964dbd-7f42-4e43-8f7b-6316add8da1a None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:18.775503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81964dbd-7f42-4e43-8f7b-6316add8da1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:18.775808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81964dbd-7f42-4e43-8f7b-6316add8da1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:18.777227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81964dbd-7f42-4e43-8f7b-6316add8da1a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:18.791282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 348/718] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:18.796743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034acc-69d6-4c06-b06c-ef043e11f021 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:18.828094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:18.887135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956fd03-055e-4c6a-9b74-126f3d990bce None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:18.887672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956fd03-055e-4c6a-9b74-126f3d990bce None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:18.887672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956fd03-055e-4c6a-9b74-126f3d990bce None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:18.889260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 349/719] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:41:18.894293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c8db9-72b5-49cb-852a-15790642ba67 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:18.894577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c8db9-72b5-49cb-852a-15790642ba67 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:18.894853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c8db9-72b5-49cb-852a-15790642ba67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:18.912607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034acc-69d6-4c06-b06c-ef043e11f021 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:18.912824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-9b2c8db9-72b5-49cb-852a-15790642ba67 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:18.914347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9b2c8db9-72b5-49cb-852a-15790642ba67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:18.975451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b08e6-9f9f-4a33-8b3a-2e580bd72dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:18.976179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b08e6-9f9f-4a33-8b3a-2e580bd72dd1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:18.976901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b08e6-9f9f-4a33-8b3a-2e580bd72dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:18.977175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b08e6-9f9f-4a33-8b3a-2e580bd72dd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:18.977994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b08e6-9f9f-4a33-8b3a-2e580bd72dd1 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=54b05cbc41b844d8932ba4c15d960289) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:18.978301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b08e6-9f9f-4a33-8b3a-2e580bd72dd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:18.979894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b08e6-9f9f-4a33-8b3a-2e580bd72dd1 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:18.989952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 350/720] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 16:41:18.991811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 371/721] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:18 2022] GET /identity/v3/role_assignments?user.id=54b05cbc41b844d8932ba4c15d960289&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:19.014817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c8db9-72b5-49cb-852a-15790642ba67 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:19.100053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:19.124681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c8db9-72b5-49cb-852a-15790642ba67 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:19.162872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a251db3-fa8a-48eb-bb4b-4d10d009bd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:19.163568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a251db3-fa8a-48eb-bb4b-4d10d009bd79 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:19.163826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a251db3-fa8a-48eb-bb4b-4d10d009bd79 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:19.164100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a251db3-fa8a-48eb-bb4b-4d10d009bd79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:19.175377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a251db3-fa8a-48eb-bb4b-4d10d009bd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-9a251db3-fa8a-48eb-bb4b-4d10d009bd79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:19.180706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:19.181647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:19.182507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 351/722] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:19.188172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b08e6-9f9f-4a33-8b3a-2e580bd72dd1 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:19.311151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:19.324327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b08e6-9f9f-4a33-8b3a-2e580bd72dd1 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:19.360033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879d5fd-6c06-4448-b227-6da1be133f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:19.360684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879d5fd-6c06-4448-b227-6da1be133f49 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:19.361009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879d5fd-6c06-4448-b227-6da1be133f49 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:19.361278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879d5fd-6c06-4448-b227-6da1be133f49 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:19.362151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879d5fd-6c06-4448-b227-6da1be133f49 None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:19.362456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879d5fd-6c06-4448-b227-6da1be133f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:19.363869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879d5fd-6c06-4448-b227-6da1be133f49 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:19.373951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 372/723] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:19.383130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a251db3-fa8a-48eb-bb4b-4d10d009bd79 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:19.501935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a251db3-fa8a-48eb-bb4b-4d10d009bd79 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:19.536829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07075da5-902b-4657-ad09-29fc54c8512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:19.537468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07075da5-902b-4657-ad09-29fc54c8512b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:19.537744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07075da5-902b-4657-ad09-29fc54c8512b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:19.538043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07075da5-902b-4657-ad09-29fc54c8512b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:19.545356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:19.549122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07075da5-902b-4657-ad09-29fc54c8512b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:19.550593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07075da5-902b-4657-ad09-29fc54c8512b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-07075da5-902b-4657-ad09-29fc54c8512b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:19.567743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:19.568714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:19.569598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:19.569598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:19.569598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:19.569598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 352/724] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:19.574988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879d5fd-6c06-4448-b227-6da1be133f49 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:19.709599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879d5fd-6c06-4448-b227-6da1be133f49 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:19.758514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48623c-33dc-450d-8189-95e5ea0bed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:19.759134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48623c-33dc-450d-8189-95e5ea0bed2f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:19.759464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48623c-33dc-450d-8189-95e5ea0bed2f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:19.759747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48623c-33dc-450d-8189-95e5ea0bed2f None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:19.760400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48623c-33dc-450d-8189-95e5ea0bed2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:19.760749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48623c-33dc-450d-8189-95e5ea0bed2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:19.762149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48623c-33dc-450d-8189-95e5ea0bed2f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:19.768048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52356d-3dd2-49df-b646-e00d56c35f57 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:19.768322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52356d-3dd2-49df-b646-e00d56c35f57 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:19.768586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52356d-3dd2-49df-b646-e00d56c35f57 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:19.769948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 353/725] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:19.775811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b3cc-79c6-4790-ab9e-058629df5db6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:19.776096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b3cc-79c6-4790-ab9e-058629df5db6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:19.776355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b3cc-79c6-4790-ab9e-058629df5db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:19.791692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 373/726] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:19.804542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48623c-33dc-450d-8189-95e5ea0bed2f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:19.810746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d971b3cc-79c6-4790-ab9e-058629df5db6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:19.816497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d971b3cc-79c6-4790-ab9e-058629df5db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:19.825206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:19.978457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48623c-33dc-450d-8189-95e5ea0bed2f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:20.004558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 354/727] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:20.015654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c515-021a-4927-bce9-990db5938d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:20.016285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c515-021a-4927-bce9-990db5938d3e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.016569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c515-021a-4927-bce9-990db5938d3e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.016892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c515-021a-4927-bce9-990db5938d3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.024765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0c515-021a-4927-bce9-990db5938d3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:20.026248 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0c515-021a-4927-bce9-990db5938d3e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:20.039616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971b3cc-79c6-4790-ab9e-058629df5db6 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-03a0c515-021a-4927-bce9-990db5938d3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:20.041895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:20.042846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:20.043813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 374/728] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:41:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:20.049455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c515-021a-4927-bce9-990db5938d3e None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:20.136644 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:20.159869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971b3cc-79c6-4790-ab9e-058629df5db6 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:20.174816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c515-021a-4927-bce9-990db5938d3e None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:20.183036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:20.198669 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff233ba3-63b1-4a82-9006-a19ef4709baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:20.199473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff233ba3-63b1-4a82-9006-a19ef4709baf None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.200016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff233ba3-63b1-4a82-9006-a19ef4709baf None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.200432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff233ba3-63b1-4a82-9006-a19ef4709baf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.212652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff233ba3-63b1-4a82-9006-a19ef4709baf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:20.215916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b5ed7-e5ea-4945-a148-7dcd1a01c89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:20.216516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5ed7-e5ea-4945-a148-7dcd1a01c89e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.216796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5ed7-e5ea-4945-a148-7dcd1a01c89e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.217069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5ed7-e5ea-4945-a148-7dcd1a01c89e None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.217843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b5ed7-e5ea-4945-a148-7dcd1a01c89e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:20.218129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b5ed7-e5ea-4945-a148-7dcd1a01c89e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-ff233ba3-63b1-4a82-9006-a19ef4709baf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:20.218538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:20.219610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:20.220612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:20.221224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b5ed7-e5ea-4945-a148-7dcd1a01c89e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:20.228095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 355/729] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:20.229775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a5435-ff47-4fb9-a904-4b4f0a5963b3 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.230224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a5435-ff47-4fb9-a904-4b4f0a5963b3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.232277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a5435-ff47-4fb9-a904-4b4f0a5963b3 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.234086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 356/730] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:20.238824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a5435-ff47-4fb9-a904-4b4f0a5963b3 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:20.238951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 375/731] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Aug 8 16:41:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:20.243480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde65d9-d9a3-48de-9b7a-d73772c09ce1 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.243769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde65d9-d9a3-48de-9b7a-d73772c09ce1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.244879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde65d9-d9a3-48de-9b7a-d73772c09ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.266366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-6fde65d9-d9a3-48de-9b7a-d73772c09ce1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:20.267719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6fde65d9-d9a3-48de-9b7a-d73772c09ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:20.348491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 376/732] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:20.349382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde65d9-d9a3-48de-9b7a-d73772c09ce1 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:20.390601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a5435-ff47-4fb9-a904-4b4f0a5963b3 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:20.417566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:20.436313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1da1b3-3dba-4d62-ac30-77e9fa3ad869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:20.437052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1da1b3-3dba-4d62-ac30-77e9fa3ad869 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.437752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1da1b3-3dba-4d62-ac30-77e9fa3ad869 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.437752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1da1b3-3dba-4d62-ac30-77e9fa3ad869 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.439103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1da1b3-3dba-4d62-ac30-77e9fa3ad869 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:20.439411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1da1b3-3dba-4d62-ac30-77e9fa3ad869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:20.441218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1da1b3-3dba-4d62-ac30-77e9fa3ad869 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:20.458554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 357/733] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:20.458554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1da1b3-3dba-4d62-ac30-77e9fa3ad869 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:20.470658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde65d9-d9a3-48de-9b7a-d73772c09ce1 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:20.508566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b9a6c-bf13-4d51-980c-42b3f58dea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:20.509202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b9a6c-bf13-4d51-980c-42b3f58dea22 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.509480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b9a6c-bf13-4d51-980c-42b3f58dea22 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.509751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b9a6c-bf13-4d51-980c-42b3f58dea22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.513223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:20.587191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1da1b3-3dba-4d62-ac30-77e9fa3ad869 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:20.598166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b9a6c-bf13-4d51-980c-42b3f58dea22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:20.599908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b9a6c-bf13-4d51-980c-42b3f58dea22 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:20.639608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e2523-865d-4a6e-acb4-b82ba95ea5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:20.641131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e2523-865d-4a6e-acb4-b82ba95ea5bc None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.641604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e2523-865d-4a6e-acb4-b82ba95ea5bc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.642115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e2523-865d-4a6e-acb4-b82ba95ea5bc None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.643385 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e2523-865d-4a6e-acb4-b82ba95ea5bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:20.645617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e2523-865d-4a6e-acb4-b82ba95ea5bc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:20.660358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:20.703931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 377/734] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 16:41:20.709056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b9a6c-bf13-4d51-980c-42b3f58dea22 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:20.723079 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 358/735] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:41:20 2022] POST /identity/v3/services => generated 251 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:20.724938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e2523-865d-4a6e-acb4-b82ba95ea5bc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:20.839636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b9a6c-bf13-4d51-980c-42b3f58dea22 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:20.872482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:20.887891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e2523-865d-4a6e-acb4-b82ba95ea5bc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:20.904668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fec3d9-fe26-401b-998d-9bd804ecb37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:20.905963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fec3d9-fe26-401b-998d-9bd804ecb37c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.906619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fec3d9-fe26-401b-998d-9bd804ecb37c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.907032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fec3d9-fe26-401b-998d-9bd804ecb37c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.922812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fec3d9-fe26-401b-998d-9bd804ecb37c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:20.924682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fec3d9-fe26-401b-998d-9bd804ecb37c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:20.928909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da36c60-158a-48ac-b2c0-765a008b321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:20.929957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da36c60-158a-48ac-b2c0-765a008b321f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.930435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da36c60-158a-48ac-b2c0-765a008b321f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.930895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da36c60-158a-48ac-b2c0-765a008b321f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-87fec3d9-fe26-401b-998d-9bd804ecb37c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:20.942814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:20.951481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:20.952451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:20.952451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:20.952451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:20.952621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da36c60-158a-48ac-b2c0-765a008b321f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:20.952695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 378/736] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:20.952695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17036b1e-1be9-449b-8f2d-42f84e621df9 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.952695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17036b1e-1be9-449b-8f2d-42f84e621df9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.952695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17036b1e-1be9-449b-8f2d-42f84e621df9 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:20.952935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 379/737] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:20.954484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17036b1e-1be9-449b-8f2d-42f84e621df9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:20.959865 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da36c60-158a-48ac-b2c0-765a008b321f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:20.970269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-2da36c60-158a-48ac-b2c0-765a008b321f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:20.981921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:20.982881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:20.983751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:20.983751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:20.983751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:20.986344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 359/738] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:20.988377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61708112-d6b4-44fd-9096-e27d7514889a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:20.989170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61708112-d6b4-44fd-9096-e27d7514889a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:20.989637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61708112-d6b4-44fd-9096-e27d7514889a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.012568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-61708112-d6b4-44fd-9096-e27d7514889a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:21.013874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-61708112-d6b4-44fd-9096-e27d7514889a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:21.083046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17036b1e-1be9-449b-8f2d-42f84e621df9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:21.108720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 360/739] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 16:41:21.109730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61708112-d6b4-44fd-9096-e27d7514889a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:21.132001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb4fde-579d-4077-8f9a-24beae901474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:21.132674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb4fde-579d-4077-8f9a-24beae901474 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.132948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb4fde-579d-4077-8f9a-24beae901474 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.133210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb4fde-579d-4077-8f9a-24beae901474 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.133885 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb4fde-579d-4077-8f9a-24beae901474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:21.134184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb4fde-579d-4077-8f9a-24beae901474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:21.135527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb4fde-579d-4077-8f9a-24beae901474 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:21.148421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 380/740] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:21.149320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb4fde-579d-4077-8f9a-24beae901474 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:21.219638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:21.240187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:21.249164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61708112-d6b4-44fd-9096-e27d7514889a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:21.277014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb4fde-579d-4077-8f9a-24beae901474 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:21.294543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b5747-b917-450c-8cd6-3b1952868b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:21.295686 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b5747-b917-450c-8cd6-3b1952868b88 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.296404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b5747-b917-450c-8cd6-3b1952868b88 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.296847 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b5747-b917-450c-8cd6-3b1952868b88 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.297741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b5747-b917-450c-8cd6-3b1952868b88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:21.298298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b5747-b917-450c-8cd6-3b1952868b88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:21.300583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b5747-b917-450c-8cd6-3b1952868b88 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:21.319017 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 361/741] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:21.319969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b5747-b917-450c-8cd6-3b1952868b88 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:21.321626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03301f-5aa0-4c0b-8201-7bcb8a2ab5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:21.323122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03301f-5aa0-4c0b-8201-7bcb8a2ab5f9 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.323597 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03301f-5aa0-4c0b-8201-7bcb8a2ab5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.324954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03301f-5aa0-4c0b-8201-7bcb8a2ab5f9 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.325610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03301f-5aa0-4c0b-8201-7bcb8a2ab5f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:21.327078 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03301f-5aa0-4c0b-8201-7bcb8a2ab5f9 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:21.404335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 381/742] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:41:21 2022] POST /identity/v3/services => generated 240 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:21.405257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03301f-5aa0-4c0b-8201-7bcb8a2ab5f9 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:21.513865 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b5747-b917-450c-8cd6-3b1952868b88 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:21.552776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03301f-5aa0-4c0b-8201-7bcb8a2ab5f9 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:21.564780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d34d75-9ee7-4c2d-8278-4f6c74973c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:21.566234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d34d75-9ee7-4c2d-8278-4f6c74973c9d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.566663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d34d75-9ee7-4c2d-8278-4f6c74973c9d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.567478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d34d75-9ee7-4c2d-8278-4f6c74973c9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.579184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d34d75-9ee7-4c2d-8278-4f6c74973c9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:21.581171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d34d75-9ee7-4c2d-8278-4f6c74973c9d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:21.588572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:21.590032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-f5d34d75-9ee7-4c2d-8278-4f6c74973c9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:21.600421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:21.602066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:21.602965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:21.603478 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0efbc-8aa5-4b02-8b9d-b92c1fa16ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:21.603478 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0efbc-8aa5-4b02-8b9d-b92c1fa16ee3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.603629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 362/743] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:21.605096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0efbc-8aa5-4b02-8b9d-b92c1fa16ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.605205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd790f3e-cc7b-4ad6-a521-53798f1e674f None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.605288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0efbc-8aa5-4b02-8b9d-b92c1fa16ee3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.605358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd790f3e-cc7b-4ad6-a521-53798f1e674f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.605358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd790f3e-cc7b-4ad6-a521-53798f1e674f None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.611202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 363/744] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:21.614841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd790f3e-cc7b-4ad6-a521-53798f1e674f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:21.617272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0efbc-8aa5-4b02-8b9d-b92c1fa16ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:21.618339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0efbc-8aa5-4b02-8b9d-b92c1fa16ee3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-19b0efbc-8aa5-4b02-8b9d-b92c1fa16ee3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:21.630511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:21.631468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:21.632896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:21.632896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:21.632896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:21.632896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:21.632896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:21.632896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:21.632896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:21.632896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:21.632896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 382/745] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:21.633903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a82ad-8596-4ebb-ba99-dce71a2ca782 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.635261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a82ad-8596-4ebb-ba99-dce71a2ca782 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.635261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a82ad-8596-4ebb-ba99-dce71a2ca782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.655840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-031a82ad-8596-4ebb-ba99-dce71a2ca782 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:21.656959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-031a82ad-8596-4ebb-ba99-dce71a2ca782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:21.738772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 383/746] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:21.739826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a82ad-8596-4ebb-ba99-dce71a2ca782 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:21.759404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd790f3e-cc7b-4ad6-a521-53798f1e674f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:21.808009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cea76-b9ee-4e39-a40f-08a1ed9f8bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:21.808821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cea76-b9ee-4e39-a40f-08a1ed9f8bb0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.809099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cea76-b9ee-4e39-a40f-08a1ed9f8bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.809374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cea76-b9ee-4e39-a40f-08a1ed9f8bb0 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.810795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cea76-b9ee-4e39-a40f-08a1ed9f8bb0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:21.811107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cea76-b9ee-4e39-a40f-08a1ed9f8bb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:21.812822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cea76-b9ee-4e39-a40f-08a1ed9f8bb0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:21.836006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 364/747] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Aug 8 16:41:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:21.836837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cea76-b9ee-4e39-a40f-08a1ed9f8bb0 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:21.856761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a82ad-8596-4ebb-ba99-dce71a2ca782 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:21.874727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:21.899415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420c0cb-4ae2-4082-9bac-7829055a8c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:21.900290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420c0cb-4ae2-4082-9bac-7829055a8c43 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.900574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420c0cb-4ae2-4082-9bac-7829055a8c43 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.900843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420c0cb-4ae2-4082-9bac-7829055a8c43 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:21.901855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420c0cb-4ae2-4082-9bac-7829055a8c43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:21.902525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420c0cb-4ae2-4082-9bac-7829055a8c43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:21.903971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420c0cb-4ae2-4082-9bac-7829055a8c43 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:21.911109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:21.922196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 384/748] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:21.923359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420c0cb-4ae2-4082-9bac-7829055a8c43 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:21.957297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cea76-b9ee-4e39-a40f-08a1ed9f8bb0 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:21.996190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da15c4-a6ed-49e5-a64d-0c5eb5010407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:21.997038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da15c4-a6ed-49e5-a64d-0c5eb5010407 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:21.997374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da15c4-a6ed-49e5-a64d-0c5eb5010407 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:21.997700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da15c4-a6ed-49e5-a64d-0c5eb5010407 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:22.009247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da15c4-a6ed-49e5-a64d-0c5eb5010407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a3da15c4-a6ed-49e5-a64d-0c5eb5010407 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:22.014407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:22.016765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 365/749] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:22.018813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da15c4-a6ed-49e5-a64d-0c5eb5010407 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:22.054680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420c0cb-4ae2-4082-9bac-7829055a8c43 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:22.096274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb4ef1-d8b5-471d-be50-4f40fc08fbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:22.097354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4ef1-d8b5-471d-be50-4f40fc08fbc7 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:22.097950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4ef1-d8b5-471d-be50-4f40fc08fbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:22.098437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4ef1-d8b5-471d-be50-4f40fc08fbc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:22.130085 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da15c4-a6ed-49e5-a64d-0c5eb5010407 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:22.149036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:22.149888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:22.167293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7298848-e36b-40a8-8627-9a2f54d30156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:22.167780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7298848-e36b-40a8-8627-9a2f54d30156 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:22.168053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7298848-e36b-40a8-8627-9a2f54d30156 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:22.168317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7298848-e36b-40a8-8627-9a2f54d30156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:22.169063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7298848-e36b-40a8-8627-9a2f54d30156 None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=19cf8bdae18e46ac8217c021765a2a78) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:22.169347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7298848-e36b-40a8-8627-9a2f54d30156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:22.170882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7298848-e36b-40a8-8627-9a2f54d30156 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:22.175526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb4ef1-d8b5-471d-be50-4f40fc08fbc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:22.176889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 366/750] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:22 2022] GET /identity/v3/role_assignments?user.id=19cf8bdae18e46ac8217c021765a2a78&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:22.177454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb4ef1-d8b5-471d-be50-4f40fc08fbc7 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:22.177694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7298848-e36b-40a8-8627-9a2f54d30156 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:22.291983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 385/751] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:41:22.297979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb4ef1-d8b5-471d-be50-4f40fc08fbc7 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:22.319479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7298848-e36b-40a8-8627-9a2f54d30156 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:22.393559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450da93-c5c3-43e1-b2a3-5280326e61bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:22.394142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450da93-c5c3-43e1-b2a3-5280326e61bf None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:22.394416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450da93-c5c3-43e1-b2a3-5280326e61bf None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:22.394683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450da93-c5c3-43e1-b2a3-5280326e61bf None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:22.395356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450da93-c5c3-43e1-b2a3-5280326e61bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:22.395655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450da93-c5c3-43e1-b2a3-5280326e61bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:22.397525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450da93-c5c3-43e1-b2a3-5280326e61bf None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:22.406428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:22.417737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 367/752] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:22.426315 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb4ef1-d8b5-471d-be50-4f40fc08fbc7 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:22.432325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450da93-c5c3-43e1-b2a3-5280326e61bf None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:22.474268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5330a-4edd-4d6b-9590-c932abfe3b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:22.475479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5330a-4edd-4d6b-9590-c932abfe3b75 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:22.477388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5330a-4edd-4d6b-9590-c932abfe3b75 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:22.477843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5330a-4edd-4d6b-9590-c932abfe3b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:22.498087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5330a-4edd-4d6b-9590-c932abfe3b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:22.498490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:22.499557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5330a-4edd-4d6b-9590-c932abfe3b75 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-b9f5330a-4edd-4d6b-9590-c932abfe3b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:22.522467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:22.523535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:22.524752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:22.524752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:22.524752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:22.528727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 386/753] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:22.529947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5330a-4edd-4d6b-9590-c932abfe3b75 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:22.661716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450da93-c5c3-43e1-b2a3-5280326e61bf None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:22.665699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5330a-4edd-4d6b-9590-c932abfe3b75 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:22.721499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:22.730962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e4e49-cf21-4ad6-a83a-4bc8b22a6bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:22.731646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e4e49-cf21-4ad6-a83a-4bc8b22a6bfd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:22.731917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e4e49-cf21-4ad6-a83a-4bc8b22a6bfd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:22.735319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e4e49-cf21-4ad6-a83a-4bc8b22a6bfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:22.735436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c40062-73e1-4760-9aa0-143833274585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:22.735436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c40062-73e1-4760-9aa0-143833274585 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:22.735436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c40062-73e1-4760-9aa0-143833274585 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:22.735771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c40062-73e1-4760-9aa0-143833274585 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:22.736377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c40062-73e1-4760-9aa0-143833274585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:22.736675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c40062-73e1-4760-9aa0-143833274585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:22.738047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c40062-73e1-4760-9aa0-143833274585 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:22.757730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e4e49-cf21-4ad6-a83a-4bc8b22a6bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:22.759213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e4e49-cf21-4ad6-a83a-4bc8b22a6bfd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:22.764864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 387/754] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:22.770578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c40062-73e1-4760-9aa0-143833274585 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-af8e4e49-cf21-4ad6-a83a-4bc8b22a6bfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:22.786744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:22.787724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:22.789247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:22.789247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:22.789247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:22.790389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 368/755] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:22.798743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e4e49-cf21-4ad6-a83a-4bc8b22a6bfd None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:22.933411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c40062-73e1-4760-9aa0-143833274585 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:22.964036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:22.975533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e4e49-cf21-4ad6-a83a-4bc8b22a6bfd None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:22.999561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09368d-9567-461c-9aaa-8f27ad8a2691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.000511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09368d-9567-461c-9aaa-8f27ad8a2691 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.000930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09368d-9567-461c-9aaa-8f27ad8a2691 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.001527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09368d-9567-461c-9aaa-8f27ad8a2691 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.012356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09368d-9567-461c-9aaa-8f27ad8a2691 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.016810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09368d-9567-461c-9aaa-8f27ad8a2691 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-eb09368d-9567-461c-9aaa-8f27ad8a2691 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:23.034085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:23.035159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:23.036236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:23.036236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:23.036236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:23.036236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:23.036236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:23.036236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:23.036236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:23.036236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:23.045679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007314a-2ca7-4399-8caa-191a5b2edfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.046857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007314a-2ca7-4399-8caa-191a5b2edfff None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.047773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 388/756] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:23.048219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007314a-2ca7-4399-8caa-191a5b2edfff None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.048635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007314a-2ca7-4399-8caa-191a5b2edfff None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.049484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007314a-2ca7-4399-8caa-191a5b2edfff None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:23.049992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007314a-2ca7-4399-8caa-191a5b2edfff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.051558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007314a-2ca7-4399-8caa-191a5b2edfff None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.053095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09368d-9567-461c-9aaa-8f27ad8a2691 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:23.081711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 369/757] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:23.087928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007314a-2ca7-4399-8caa-191a5b2edfff None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:23.133391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:23.189400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09368d-9567-461c-9aaa-8f27ad8a2691 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:23.221526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007314a-2ca7-4399-8caa-191a5b2edfff None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:23.228138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264a2f-0503-4521-92a1-81049738ca04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.229014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264a2f-0503-4521-92a1-81049738ca04 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.229490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264a2f-0503-4521-92a1-81049738ca04 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.229984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264a2f-0503-4521-92a1-81049738ca04 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.230959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37264a2f-0503-4521-92a1-81049738ca04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:23.231473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37264a2f-0503-4521-92a1-81049738ca04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.233174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37264a2f-0503-4521-92a1-81049738ca04 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.248130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 389/758] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:23.249979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335cef5-1b58-4b77-9402-5357766845d8 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.250263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335cef5-1b58-4b77-9402-5357766845d8 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.250534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335cef5-1b58-4b77-9402-5357766845d8 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.253856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 390/759] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:23.257535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335cef5-1b58-4b77-9402-5357766845d8 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:23.264267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b34ca-86f8-47c6-a9a0-0f6d8f9ae5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.265216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b34ca-86f8-47c6-a9a0-0f6d8f9ae5bd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.265722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b34ca-86f8-47c6-a9a0-0f6d8f9ae5bd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.266230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b34ca-86f8-47c6-a9a0-0f6d8f9ae5bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.274994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b34ca-86f8-47c6-a9a0-0f6d8f9ae5bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.276777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b34ca-86f8-47c6-a9a0-0f6d8f9ae5bd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b20b34ca-86f8-47c6-a9a0-0f6d8f9ae5bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:23.289877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:23.290826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:23.291766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:23.294435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 370/760] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:23.296029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eab79c-e9a5-401b-8f10-ec1ece1bb948 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.296576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eab79c-e9a5-401b-8f10-ec1ece1bb948 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.297142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eab79c-e9a5-401b-8f10-ec1ece1bb948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.316800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-39eab79c-e9a5-401b-8f10-ec1ece1bb948 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:23.318472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-39eab79c-e9a5-401b-8f10-ec1ece1bb948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:23.330897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:23.368848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335cef5-1b58-4b77-9402-5357766845d8 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:23.379562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:23.392946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 371/761] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:23.394311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eab79c-e9a5-401b-8f10-ec1ece1bb948 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:23.419973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bb7a9-d6f0-4744-9a65-d9ec6cc2aa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.420555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bb7a9-d6f0-4744-9a65-d9ec6cc2aa8b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.420833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bb7a9-d6f0-4744-9a65-d9ec6cc2aa8b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.421098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bb7a9-d6f0-4744-9a65-d9ec6cc2aa8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.426120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bb7a9-d6f0-4744-9a65-d9ec6cc2aa8b None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=92130daa3c3a49c2adc597dc563b2973) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:23.426447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bb7a9-d6f0-4744-9a65-d9ec6cc2aa8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.427876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bb7a9-d6f0-4744-9a65-d9ec6cc2aa8b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.438699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 391/762] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/role_assignments?user.id=92130daa3c3a49c2adc597dc563b2973&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:23.440811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bb7a9-d6f0-4744-9a65-d9ec6cc2aa8b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:23.511615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eab79c-e9a5-401b-8f10-ec1ece1bb948 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:23.554007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8f466-06df-4799-ade7-0d93b40f4f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.555053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8f466-06df-4799-ade7-0d93b40f4f35 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.555533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8f466-06df-4799-ade7-0d93b40f4f35 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.556059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8f466-06df-4799-ade7-0d93b40f4f35 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.556177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bb7a9-d6f0-4744-9a65-d9ec6cc2aa8b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:23.557216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8f466-06df-4799-ade7-0d93b40f4f35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:23.557856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8f466-06df-4799-ade7-0d93b40f4f35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.559652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8f466-06df-4799-ade7-0d93b40f4f35 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.580998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 372/763] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:23.592116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8f466-06df-4799-ade7-0d93b40f4f35 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:23.597443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:23.598733 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac379438-b4a2-4639-b701-82790f2cd9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.599398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac379438-b4a2-4639-b701-82790f2cd9db None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.599675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac379438-b4a2-4639-b701-82790f2cd9db None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.599940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac379438-b4a2-4639-b701-82790f2cd9db None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.600518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac379438-b4a2-4639-b701-82790f2cd9db None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.602529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac379438-b4a2-4639-b701-82790f2cd9db None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-ac379438-b4a2-4639-b701-82790f2cd9db None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:23.614641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:23.615704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:23.617085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 392/764] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:23.622071 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac379438-b4a2-4639-b701-82790f2cd9db None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:23.660087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:23.721470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8f466-06df-4799-ade7-0d93b40f4f35 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:23.731914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac379438-b4a2-4639-b701-82790f2cd9db None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:23.759726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3b607-42eb-44fa-bb88-b00d6715c08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.760789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3b607-42eb-44fa-bb88-b00d6715c08c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.761358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3b607-42eb-44fa-bb88-b00d6715c08c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.761880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3b607-42eb-44fa-bb88-b00d6715c08c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.765927 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c5389-8dfe-4de7-8017-62b2b72a0fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.766525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c5389-8dfe-4de7-8017-62b2b72a0fa2 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.766799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c5389-8dfe-4de7-8017-62b2b72a0fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.767125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c5389-8dfe-4de7-8017-62b2b72a0fa2 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.767750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c5389-8dfe-4de7-8017-62b2b72a0fa2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:23.768046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c5389-8dfe-4de7-8017-62b2b72a0fa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.769399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c5389-8dfe-4de7-8017-62b2b72a0fa2 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.778972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 393/765] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:23.778972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5754d-ac6f-4e30-b169-e3a08e3a4a61 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.778972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5754d-ac6f-4e30-b169-e3a08e3a4a61 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.778972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5754d-ac6f-4e30-b169-e3a08e3a4a61 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.778972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 394/766] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:23.779958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5754d-ac6f-4e30-b169-e3a08e3a4a61 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:23.780961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3b607-42eb-44fa-bb88-b00d6715c08c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.782823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3b607-42eb-44fa-bb88-b00d6715c08c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-22e3b607-42eb-44fa-bb88-b00d6715c08c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:23.801320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:23.802319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:23.803226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:23.803226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:23.803226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:23.806234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 373/767] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:23.808288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378788a-c5a6-46c4-9d3e-d901a034ca14 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.809016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378788a-c5a6-46c4-9d3e-d901a034ca14 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.809582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378788a-c5a6-46c4-9d3e-d901a034ca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.834919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b378788a-c5a6-46c4-9d3e-d901a034ca14 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:23.836985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b378788a-c5a6-46c4-9d3e-d901a034ca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:23.895284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:23.901667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5754d-ac6f-4e30-b169-e3a08e3a4a61 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:23.914405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:23.937143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 374/768] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:23.938289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378788a-c5a6-46c4-9d3e-d901a034ca14 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:23.942711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0e009-a381-4c45-b350-17793340a400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:23.942863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e009-a381-4c45-b350-17793340a400 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:23.943240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e009-a381-4c45-b350-17793340a400 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:23.946167 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e009-a381-4c45-b350-17793340a400 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:23.947183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e009-a381-4c45-b350-17793340a400 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:23.947552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e009-a381-4c45-b350-17793340a400 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:23.949019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e009-a381-4c45-b350-17793340a400 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:23.964437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 395/769] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:23.970356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0e009-a381-4c45-b350-17793340a400 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.068891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378788a-c5a6-46c4-9d3e-d901a034ca14 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.084358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0e009-a381-4c45-b350-17793340a400 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.105268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b38c0e-ccf8-4885-b786-6be66770ac77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.106011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b38c0e-ccf8-4885-b786-6be66770ac77 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.106350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b38c0e-ccf8-4885-b786-6be66770ac77 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.106686 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b38c0e-ccf8-4885-b786-6be66770ac77 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.107411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b38c0e-ccf8-4885-b786-6be66770ac77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:24.107768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b38c0e-ccf8-4885-b786-6be66770ac77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.109202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b38c0e-ccf8-4885-b786-6be66770ac77 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.121418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 375/770] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:24.122464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b38c0e-ccf8-4885-b786-6be66770ac77 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.122671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e54856-da0a-45bc-a8d1-fab147fac31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.123522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e54856-da0a-45bc-a8d1-fab147fac31b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.123996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e54856-da0a-45bc-a8d1-fab147fac31b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.124503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e54856-da0a-45bc-a8d1-fab147fac31b None admin] PATH_INFO: `/v3/endpoints` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.125376 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e54856-da0a-45bc-a8d1-fab147fac31b None admin] RBAC: Adding query filter params (interface=public, service_id=c93a078154bf4ef2b2b64c37bb030bd8, region_id=RegionOne) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:24.125882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e54856-da0a-45bc-a8d1-fab147fac31b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.127457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e54856-da0a-45bc-a8d1-fab147fac31b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.140581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 396/771] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Aug 8 16:41:23 2022] GET /identity/v3/endpoints?service_id=c93a078154bf4ef2b2b64c37bb030bd8&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:24.141763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e54856-da0a-45bc-a8d1-fab147fac31b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.171775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:24.172879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:24.242686 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b38c0e-ccf8-4885-b786-6be66770ac77 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.248023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e54856-da0a-45bc-a8d1-fab147fac31b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.284036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d160d-4a7e-4696-bf7b-ac9b91e41a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.284863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d160d-4a7e-4696-bf7b-ac9b91e41a57 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.285207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d160d-4a7e-4696-bf7b-ac9b91e41a57 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.285579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d160d-4a7e-4696-bf7b-ac9b91e41a57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.290101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea8c93-8253-4097-b275-cea90239c5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.291084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea8c93-8253-4097-b275-cea90239c5e8 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.291501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea8c93-8253-4097-b275-cea90239c5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.291908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea8c93-8253-4097-b275-cea90239c5e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.292626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea8c93-8253-4097-b275-cea90239c5e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.294158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea8c93-8253-4097-b275-cea90239c5e8 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.297944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d160d-4a7e-4696-bf7b-ac9b91e41a57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.297944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d160d-4a7e-4696-bf7b-ac9b91e41a57 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-39ea8c93-8253-4097-b275-cea90239c5e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:24.304291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:24.305485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:41:24.306519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:24.310709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 376/772] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:24 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:24.311566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea8c93-8253-4097-b275-cea90239c5e8 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-645d160d-4a7e-4696-bf7b-ac9b91e41a57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:24.312269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:24.315804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:24.316688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:24.316688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:24.316688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:24.316688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:24.316688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:24.316688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:24.316688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:24.316688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:24.316688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 397/773] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:24.320457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d160d-4a7e-4696-bf7b-ac9b91e41a57 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.417879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:24.422194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea8c93-8253-4097-b275-cea90239c5e8 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.431310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d160d-4a7e-4696-bf7b-ac9b91e41a57 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.434820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:24.460399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542bce5-dd44-4653-8749-c592e1c3389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.461804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542bce5-dd44-4653-8749-c592e1c3389b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.462288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542bce5-dd44-4653-8749-c592e1c3389b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.462795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542bce5-dd44-4653-8749-c592e1c3389b None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.463740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542bce5-dd44-4653-8749-c592e1c3389b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.465971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542bce5-dd44-4653-8749-c592e1c3389b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.470051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301174de-789f-4c80-9f0d-1420bb809680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.470784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301174de-789f-4c80-9f0d-1420bb809680 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.471161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301174de-789f-4c80-9f0d-1420bb809680 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.471530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301174de-789f-4c80-9f0d-1420bb809680 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.472309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301174de-789f-4c80-9f0d-1420bb809680 None admin] RBAC: Adding query filter params (name=network) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:24.472464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 377/774] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Aug 8 16:41:24 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:24.472795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301174de-789f-4c80-9f0d-1420bb809680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.473587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542bce5-dd44-4653-8749-c592e1c3389b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.474523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301174de-789f-4c80-9f0d-1420bb809680 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.484695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 398/775] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:24.486281 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301174de-789f-4c80-9f0d-1420bb809680 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.606021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542bce5-dd44-4653-8749-c592e1c3389b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.640331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301174de-789f-4c80-9f0d-1420bb809680 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.645390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb05c7-8539-4454-8b46-f8a480c04afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.646308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb05c7-8539-4454-8b46-f8a480c04afd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.646735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb05c7-8539-4454-8b46-f8a480c04afd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.647174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb05c7-8539-4454-8b46-f8a480c04afd None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.648585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb05c7-8539-4454-8b46-f8a480c04afd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:24.649046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb05c7-8539-4454-8b46-f8a480c04afd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.650656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb05c7-8539-4454-8b46-f8a480c04afd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.664530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 378/776] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:24.665923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef40a8-0e28-44c6-9756-0845be7fc399 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.666298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef40a8-0e28-44c6-9756-0845be7fc399 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.666662 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef40a8-0e28-44c6-9756-0845be7fc399 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.668266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 379/777] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:24.672907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef40a8-0e28-44c6-9756-0845be7fc399 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.684125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b26889-bf02-4fb6-9cb2-4b544bc96e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.684731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b26889-bf02-4fb6-9cb2-4b544bc96e30 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.685020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b26889-bf02-4fb6-9cb2-4b544bc96e30 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.685291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b26889-bf02-4fb6-9cb2-4b544bc96e30 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.686023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b26889-bf02-4fb6-9cb2-4b544bc96e30 None admin] RBAC: Adding query filter params (type=network) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:24.686329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b26889-bf02-4fb6-9cb2-4b544bc96e30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.687701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b26889-bf02-4fb6-9cb2-4b544bc96e30 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.695620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 399/778] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:24.697066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00301e-9270-4e5d-a1a3-3f55fc0aba45 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.697373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00301e-9270-4e5d-a1a3-3f55fc0aba45 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.697602 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00301e-9270-4e5d-a1a3-3f55fc0aba45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.717706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9a00301e-9270-4e5d-a1a3-3f55fc0aba45 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:24.719353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9a00301e-9270-4e5d-a1a3-3f55fc0aba45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:24.731463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:24.731634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:24.794023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef40a8-0e28-44c6-9756-0845be7fc399 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.808541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 400/779] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:24.809409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00301e-9270-4e5d-a1a3-3f55fc0aba45 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.844565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d313d3a-98b2-46bd-9f07-692205178045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.845272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d313d3a-98b2-46bd-9f07-692205178045 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.845549 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d313d3a-98b2-46bd-9f07-692205178045 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.845833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d313d3a-98b2-46bd-9f07-692205178045 None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/54b05cbc41b844d8932ba4c15d960289/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.847987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d313d3a-98b2-46bd-9f07-692205178045 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=54b05cbc41b844d8932ba4c15d960289, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.849397 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d313d3a-98b2-46bd-9f07-692205178045 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.916109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00301e-9270-4e5d-a1a3-3f55fc0aba45 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:24.916490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 380/780] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:41:24 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/54b05cbc41b844d8932ba4c15d960289/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:41:24.917246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d313d3a-98b2-46bd-9f07-692205178045 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.954174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bef282-2cdb-4a53-bcd8-c71eb1573d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:24.954796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bef282-2cdb-4a53-bcd8-c71eb1573d75 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:24.955127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bef282-2cdb-4a53-bcd8-c71eb1573d75 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:24.955400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bef282-2cdb-4a53-bcd8-c71eb1573d75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:24.956183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bef282-2cdb-4a53-bcd8-c71eb1573d75 None admin] RBAC: Adding query filter params (interface=public, service_id=2d5b599fd7f0469fb287175a76f10b70, region_id=RegionOne) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:24.956491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bef282-2cdb-4a53-bcd8-c71eb1573d75 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:24.958047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bef282-2cdb-4a53-bcd8-c71eb1573d75 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:24.965111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 401/781] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Aug 8 16:41:24 2022] GET /identity/v3/endpoints?service_id=2d5b599fd7f0469fb287175a76f10b70&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:24.973822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bef282-2cdb-4a53-bcd8-c71eb1573d75 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:24.998467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:25.038909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:25.045101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d313d3a-98b2-46bd-9f07-692205178045 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:25.091246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c43ce8-3a47-42a4-92d6-4234a834847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:25.091966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c43ce8-3a47-42a4-92d6-4234a834847e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:25.092158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c43ce8-3a47-42a4-92d6-4234a834847e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:25.092425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c43ce8-3a47-42a4-92d6-4234a834847e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:25.103894 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c43ce8-3a47-42a4-92d6-4234a834847e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-30c43ce8-3a47-42a4-92d6-4234a834847e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:25.109233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:25.110175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:25.110981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 381/782] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:25.118583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c43ce8-3a47-42a4-92d6-4234a834847e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:25.130917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bef282-2cdb-4a53-bcd8-c71eb1573d75 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:25.166677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257dcf2-ced7-4f8f-8fa8-8c042ec1b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:25.167334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257dcf2-ced7-4f8f-8fa8-8c042ec1b719 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:25.167612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257dcf2-ced7-4f8f-8fa8-8c042ec1b719 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:25.167883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257dcf2-ced7-4f8f-8fa8-8c042ec1b719 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:25.168494 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257dcf2-ced7-4f8f-8fa8-8c042ec1b719 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:25.169900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257dcf2-ced7-4f8f-8fa8-8c042ec1b719 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:25.176320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 402/783] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Aug 8 16:41:24 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:25.237572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c43ce8-3a47-42a4-92d6-4234a834847e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:25.275240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dfba7-7366-42f2-bb7d-89e36649ae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:25.276300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dfba7-7366-42f2-bb7d-89e36649ae45 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:25.276717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dfba7-7366-42f2-bb7d-89e36649ae45 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:25.277122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dfba7-7366-42f2-bb7d-89e36649ae45 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:25.278040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7dfba7-7366-42f2-bb7d-89e36649ae45 None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:25.278479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7dfba7-7366-42f2-bb7d-89e36649ae45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:25.279984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7dfba7-7366-42f2-bb7d-89e36649ae45 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:25.296099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 382/784] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:25.301761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257dcf2-ced7-4f8f-8fa8-8c042ec1b719 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:25.408747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:25.442069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257dcf2-ced7-4f8f-8fa8-8c042ec1b719 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:25.501960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d2653-7dd5-43a1-8828-69597947b22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:25.502866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d2653-7dd5-43a1-8828-69597947b22b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:25.503263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d2653-7dd5-43a1-8828-69597947b22b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:25.503630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d2653-7dd5-43a1-8828-69597947b22b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:25.519331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d2653-7dd5-43a1-8828-69597947b22b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:25.520766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d2653-7dd5-43a1-8828-69597947b22b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-729d2653-7dd5-43a1-8828-69597947b22b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:25.538103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:25.539047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:25.539899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:25.539899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:25.539899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:25.539899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 403/785] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:25.545103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dfba7-7366-42f2-bb7d-89e36649ae45 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:25.581314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:25.701654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dfba7-7366-42f2-bb7d-89e36649ae45 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:25.736960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f299c0-0ba4-4c97-afc4-8b5ed7351d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:25.737564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f299c0-0ba4-4c97-afc4-8b5ed7351d5e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:25.737861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f299c0-0ba4-4c97-afc4-8b5ed7351d5e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:25.738131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f299c0-0ba4-4c97-afc4-8b5ed7351d5e None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:25.738770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f299c0-0ba4-4c97-afc4-8b5ed7351d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:25.739066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f299c0-0ba4-4c97-afc4-8b5ed7351d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:25.740407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f299c0-0ba4-4c97-afc4-8b5ed7351d5e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:25.753307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 383/786] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:25.759411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d2653-7dd5-43a1-8828-69597947b22b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:25.880831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15575d2-abed-4316-a09a-e9ed728bf42a None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:25.881129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15575d2-abed-4316-a09a-e9ed728bf42a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:25.881553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15575d2-abed-4316-a09a-e9ed728bf42a None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:25.883231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 384/787] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:25.888466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9a9b5-4dec-479d-8be1-7f4aff7aa1f1 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:25.888827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9a9b5-4dec-479d-8be1-7f4aff7aa1f1 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:25.889151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9a9b5-4dec-479d-8be1-7f4aff7aa1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:25.891078 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d2653-7dd5-43a1-8828-69597947b22b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:25.921152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-15e9a9b5-4dec-479d-8be1-7f4aff7aa1f1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:25.923513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-15e9a9b5-4dec-479d-8be1-7f4aff7aa1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:25.937998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:25.956008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea3e1d-fcaa-4bb8-b2fa-08f65887a35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:25.956577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea3e1d-fcaa-4bb8-b2fa-08f65887a35b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:25.956856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea3e1d-fcaa-4bb8-b2fa-08f65887a35b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:25.957125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea3e1d-fcaa-4bb8-b2fa-08f65887a35b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:25.974845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea3e1d-fcaa-4bb8-b2fa-08f65887a35b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:25.976888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea3e1d-fcaa-4bb8-b2fa-08f65887a35b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-33ea3e1d-fcaa-4bb8-b2fa-08f65887a35b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:25.999431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:26.000425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:26.001368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:26.005907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 404/788] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:41:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:26.007729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea3e1d-fcaa-4bb8-b2fa-08f65887a35b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:26.017693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 385/789] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:26.049166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9a9b5-4dec-479d-8be1-7f4aff7aa1f1 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:26.058675 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:26.131399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea3e1d-fcaa-4bb8-b2fa-08f65887a35b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:26.166893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9a9b5-4dec-479d-8be1-7f4aff7aa1f1 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:26.167393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e285e6-dfd7-41ee-8bd8-b37d9fcb78fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:26.167837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e285e6-dfd7-41ee-8bd8-b37d9fcb78fd None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:26.168212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e285e6-dfd7-41ee-8bd8-b37d9fcb78fd None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:26.168581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e285e6-dfd7-41ee-8bd8-b37d9fcb78fd None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:26.169480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e285e6-dfd7-41ee-8bd8-b37d9fcb78fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:26.169813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e285e6-dfd7-41ee-8bd8-b37d9fcb78fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:26.171330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e285e6-dfd7-41ee-8bd8-b37d9fcb78fd None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:26.189260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 405/790] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Aug 8 16:41:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:26.195218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e285e6-dfd7-41ee-8bd8-b37d9fcb78fd None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:26.212992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb606283-d248-4072-8c2e-aa48762e9808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:26.213842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb606283-d248-4072-8c2e-aa48762e9808 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:26.214275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb606283-d248-4072-8c2e-aa48762e9808 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:26.214691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb606283-d248-4072-8c2e-aa48762e9808 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:26.233174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb606283-d248-4072-8c2e-aa48762e9808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:26.244570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-cb606283-d248-4072-8c2e-aa48762e9808 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:26.246109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:26.247121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:26.247949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 386/791] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:26.253218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb606283-d248-4072-8c2e-aa48762e9808 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:26.326020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e285e6-dfd7-41ee-8bd8-b37d9fcb78fd None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:26.333504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:26.363086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c7746-2831-4b29-9464-b6e836c93273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:26.364130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c7746-2831-4b29-9464-b6e836c93273 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:26.364624 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c7746-2831-4b29-9464-b6e836c93273 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:26.365088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c7746-2831-4b29-9464-b6e836c93273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:26.376500 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c7746-2831-4b29-9464-b6e836c93273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:26.378283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c7746-2831-4b29-9464-b6e836c93273 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-f48c7746-2831-4b29-9464-b6e836c93273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:26.394833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:26.395822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:26.396692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:26.396692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:26.396692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:26.399513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 406/792] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:26.405370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c7746-2831-4b29-9464-b6e836c93273 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:26.429962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb606283-d248-4072-8c2e-aa48762e9808 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:26.499896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a383b-406a-4682-8c6a-92d2f29ebefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:26.500512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a383b-406a-4682-8c6a-92d2f29ebefb None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:26.500785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a383b-406a-4682-8c6a-92d2f29ebefb None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:26.501051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a383b-406a-4682-8c6a-92d2f29ebefb None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:26.501717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a383b-406a-4682-8c6a-92d2f29ebefb None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:26.502039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a383b-406a-4682-8c6a-92d2f29ebefb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:26.503483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a383b-406a-4682-8c6a-92d2f29ebefb None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:26.522268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 387/793] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:26.527879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a383b-406a-4682-8c6a-92d2f29ebefb None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:26.530684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:26.537781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c7746-2831-4b29-9464-b6e836c93273 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:26.573839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f3c98-3048-4acb-a06c-0374df09342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:26.574466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f3c98-3048-4acb-a06c-0374df09342f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:26.574908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f3c98-3048-4acb-a06c-0374df09342f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:26.575036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f3c98-3048-4acb-a06c-0374df09342f None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:26.575621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f3c98-3048-4acb-a06c-0374df09342f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:26.576948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f3c98-3048-4acb-a06c-0374df09342f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:26.578394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f3c98-3048-4acb-a06c-0374df09342f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:26.602540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 407/794] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:26.608382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f3c98-3048-4acb-a06c-0374df09342f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:26.712509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a383b-406a-4682-8c6a-92d2f29ebefb None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:26.715093 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f3c98-3048-4acb-a06c-0374df09342f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:26.750123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30204b-9109-4e85-9b24-58af0f74e27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:26.750674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30204b-9109-4e85-9b24-58af0f74e27d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:26.750943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30204b-9109-4e85-9b24-58af0f74e27d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:26.751262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30204b-9109-4e85-9b24-58af0f74e27d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:26.758151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30204b-9109-4e85-9b24-58af0f74e27d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:26.759581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30204b-9109-4e85-9b24-58af0f74e27d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-ac30204b-9109-4e85-9b24-58af0f74e27d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:26.770873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:26.771762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:26.773178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:26.773178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:26.773178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:26.773178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:26.773178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:26.773178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:26.773178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:26.773178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:26.773178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 408/795] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:26.778420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30204b-9109-4e85-9b24-58af0f74e27d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:26.779375 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:26.799308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:26.804228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db81db4-44eb-4c9c-964b-3d02e518872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:26.804925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db81db4-44eb-4c9c-964b-3d02e518872d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:26.805206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db81db4-44eb-4c9c-964b-3d02e518872d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:26.805472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db81db4-44eb-4c9c-964b-3d02e518872d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:26.830821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db81db4-44eb-4c9c-964b-3d02e518872d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:26.832265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db81db4-44eb-4c9c-964b-3d02e518872d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-5db81db4-44eb-4c9c-964b-3d02e518872d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:26.860659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:26.861685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:26.862576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:26.862576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:26.862576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:26.870168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 388/796] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:26.871061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db81db4-44eb-4c9c-964b-3d02e518872d None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:26.886345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30204b-9109-4e85-9b24-58af0f74e27d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:26.921824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386c029-9dcc-4e48-8a67-d9f85baf19f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:26.922484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386c029-9dcc-4e48-8a67-d9f85baf19f9 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:26.922763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386c029-9dcc-4e48-8a67-d9f85baf19f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:26.923044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386c029-9dcc-4e48-8a67-d9f85baf19f9 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:26.923813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386c029-9dcc-4e48-8a67-d9f85baf19f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:26.924118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386c029-9dcc-4e48-8a67-d9f85baf19f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:26.925543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386c029-9dcc-4e48-8a67-d9f85baf19f9 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:26.939726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 409/797] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:26.946434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386c029-9dcc-4e48-8a67-d9f85baf19f9 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:27.052033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db81db4-44eb-4c9c-964b-3d02e518872d None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:27.053301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:27.079072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386c029-9dcc-4e48-8a67-d9f85baf19f9 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:27.089845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bbbfa-dc1a-4e69-9bd5-3007bd57ecba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:27.090817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bbbfa-dc1a-4e69-9bd5-3007bd57ecba None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.091292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bbbfa-dc1a-4e69-9bd5-3007bd57ecba None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.091776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bbbfa-dc1a-4e69-9bd5-3007bd57ecba None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.092612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bbbfa-dc1a-4e69-9bd5-3007bd57ecba None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:27.093108 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bbbfa-dc1a-4e69-9bd5-3007bd57ecba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:27.094902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bbbfa-dc1a-4e69-9bd5-3007bd57ecba None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:27.108120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 389/798] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:27.109777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed44e-9bbf-42b4-993f-8db6e6bb1266 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.110307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed44e-9bbf-42b4-993f-8db6e6bb1266 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.110772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed44e-9bbf-42b4-993f-8db6e6bb1266 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.116815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 390/799] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:27.118133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ed44e-9bbf-42b4-993f-8db6e6bb1266 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:27.123423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc522be-9ac2-4118-aa3e-83659f8403ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:27.124617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc522be-9ac2-4118-aa3e-83659f8403ec None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.125093 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc522be-9ac2-4118-aa3e-83659f8403ec None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.125555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc522be-9ac2-4118-aa3e-83659f8403ec None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/19cf8bdae18e46ac8217c021765a2a78/roles/68040d0d137e4250a000b019915a68e2` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.147333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc522be-9ac2-4118-aa3e-83659f8403ec None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=19cf8bdae18e46ac8217c021765a2a78, role_id=68040d0d137e4250a000b019915a68e2)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:27.151341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc522be-9ac2-4118-aa3e-83659f8403ec None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:27.168185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:27.226781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ed44e-9bbf-42b4-993f-8db6e6bb1266 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:27.245489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 410/800] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:41:26 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/19cf8bdae18e46ac8217c021765a2a78/roles/68040d0d137e4250a000b019915a68e2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:41:27.250185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224673ce-2733-43f4-a2b2-76e6b82261e3 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.250185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224673ce-2733-43f4-a2b2-76e6b82261e3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.250185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224673ce-2733-43f4-a2b2-76e6b82261e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.280287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-224673ce-2733-43f4-a2b2-76e6b82261e3 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:27.282420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-224673ce-2733-43f4-a2b2-76e6b82261e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:27.298258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e45b6-965b-4bd9-b3fb-293be24fcc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:27.299049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e45b6-965b-4bd9-b3fb-293be24fcc32 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.299394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e45b6-965b-4bd9-b3fb-293be24fcc32 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.299731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e45b6-965b-4bd9-b3fb-293be24fcc32 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.314969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e45b6-965b-4bd9-b3fb-293be24fcc32 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:27.316934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e45b6-965b-4bd9-b3fb-293be24fcc32 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a88e45b6-965b-4bd9-b3fb-293be24fcc32 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:27.341046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:27.342303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:27.343470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:27.344399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 391/801] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:27.350273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e45b6-965b-4bd9-b3fb-293be24fcc32 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:27.422253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 411/802] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:27.423813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce123e-b9c4-444c-be30-67b1ad2c4d93 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.424048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce123e-b9c4-444c-be30-67b1ad2c4d93 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.424386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce123e-b9c4-444c-be30-67b1ad2c4d93 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.430476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 412/803] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:27.433181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a73d3d-f834-4ffe-a275-319bc33919b7 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.433658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a73d3d-f834-4ffe-a275-319bc33919b7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.434320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a73d3d-f834-4ffe-a275-319bc33919b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.456871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a1a73d3d-f834-4ffe-a275-319bc33919b7 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:27.458217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a1a73d3d-f834-4ffe-a275-319bc33919b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:27.507515 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:27.511975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e45b6-965b-4bd9-b3fb-293be24fcc32 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:27.520285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:27.545884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 413/804] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:27.546893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a73d3d-f834-4ffe-a275-319bc33919b7 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:27.565991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b39d4-7b70-4051-963e-71b455e336bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:27.566588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b39d4-7b70-4051-963e-71b455e336bc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.566866 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b39d4-7b70-4051-963e-71b455e336bc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.567131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b39d4-7b70-4051-963e-71b455e336bc None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.567870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b39d4-7b70-4051-963e-71b455e336bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:27.568219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b39d4-7b70-4051-963e-71b455e336bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:27.570250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b39d4-7b70-4051-963e-71b455e336bc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:27.605913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 392/805] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Aug 8 16:41:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:27.605913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b39d4-7b70-4051-963e-71b455e336bc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:27.680013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a73d3d-f834-4ffe-a275-319bc33919b7 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:27.722965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56718da-f630-40b9-9002-845ce13fc6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:27.723843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56718da-f630-40b9-9002-845ce13fc6d0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.724300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56718da-f630-40b9-9002-845ce13fc6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.724723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56718da-f630-40b9-9002-845ce13fc6d0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.726084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56718da-f630-40b9-9002-845ce13fc6d0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:27.727641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56718da-f630-40b9-9002-845ce13fc6d0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-a56718da-f630-40b9-9002-845ce13fc6d0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:27.736871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:27.737858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 16:41:27.738723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:27.746076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 414/806] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Aug 8 16:41:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:27.747049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56718da-f630-40b9-9002-845ce13fc6d0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:27.747735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b39d4-7b70-4051-963e-71b455e336bc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:27.794375 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ed28f-ee2e-48db-b33b-4775e006287f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:27.795292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ed28f-ee2e-48db-b33b-4775e006287f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.795578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ed28f-ee2e-48db-b33b-4775e006287f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.795853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ed28f-ee2e-48db-b33b-4775e006287f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.801676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:27.813775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ed28f-ee2e-48db-b33b-4775e006287f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-f20ed28f-ee2e-48db-b33b-4775e006287f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:27.819221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:27.820113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:27.820978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:27.823964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 393/807] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:27.825067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ed28f-ee2e-48db-b33b-4775e006287f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:27.846485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:27.875629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56718da-f630-40b9-9002-845ce13fc6d0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:27.920794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26373634-ccb6-42c5-b3ab-97a07cae60ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:27.921448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26373634-ccb6-42c5-b3ab-97a07cae60ca None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.921726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26373634-ccb6-42c5-b3ab-97a07cae60ca None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.922022 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26373634-ccb6-42c5-b3ab-97a07cae60ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.932715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26373634-ccb6-42c5-b3ab-97a07cae60ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:27.934248 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26373634-ccb6-42c5-b3ab-97a07cae60ca None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:27.936283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ed28f-ee2e-48db-b33b-4775e006287f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-26373634-ccb6-42c5-b3ab-97a07cae60ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:27.948878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:27.954592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:27.955763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:27.955763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:27.955763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:27.955763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 415/808] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:27.955763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26373634-ccb6-42c5-b3ab-97a07cae60ca None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:27.980420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea3dc0-619d-4f18-9cdc-0013ec252341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:27.981296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea3dc0-619d-4f18-9cdc-0013ec252341 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.981577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea3dc0-619d-4f18-9cdc-0013ec252341 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.981902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea3dc0-619d-4f18-9cdc-0013ec252341 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.982539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea3dc0-619d-4f18-9cdc-0013ec252341 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:27.982854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea3dc0-619d-4f18-9cdc-0013ec252341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:27.984215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea3dc0-619d-4f18-9cdc-0013ec252341 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:27.989762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 394/809] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Aug 8 16:41:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:27.990835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc7b2a-375e-40e0-9872-bb1bb9e4bc71 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:27.991693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc7b2a-375e-40e0-9872-bb1bb9e4bc71 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:27.991961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc7b2a-375e-40e0-9872-bb1bb9e4bc71 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:27.993160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 395/810] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:27.993887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc7b2a-375e-40e0-9872-bb1bb9e4bc71 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.064097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26373634-ccb6-42c5-b3ab-97a07cae60ca None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.065747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:28.091884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:28.101525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b65fdf-3954-4e6b-a03d-c2941e933dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.107052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b65fdf-3954-4e6b-a03d-c2941e933dcf None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.107052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b65fdf-3954-4e6b-a03d-c2941e933dcf None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.107052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b65fdf-3954-4e6b-a03d-c2941e933dcf None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.107052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b65fdf-3954-4e6b-a03d-c2941e933dcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:28.107052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b65fdf-3954-4e6b-a03d-c2941e933dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.107052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b65fdf-3954-4e6b-a03d-c2941e933dcf None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.111121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc7b2a-375e-40e0-9872-bb1bb9e4bc71 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.122526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 416/811] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:28.122526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b65fdf-3954-4e6b-a03d-c2941e933dcf None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.150944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4ce1b-4d19-4167-b683-ffc1b218d275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.151647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4ce1b-4d19-4167-b683-ffc1b218d275 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.151997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4ce1b-4d19-4167-b683-ffc1b218d275 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.152339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4ce1b-4d19-4167-b683-ffc1b218d275 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.153074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4ce1b-4d19-4167-b683-ffc1b218d275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:28.153451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4ce1b-4d19-4167-b683-ffc1b218d275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.154901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4ce1b-4d19-4167-b683-ffc1b218d275 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.167752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 396/812] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:28.171027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f035ec-4a05-4f8b-8191-de0d5c93b25a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.171573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f035ec-4a05-4f8b-8191-de0d5c93b25a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.172453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f035ec-4a05-4f8b-8191-de0d5c93b25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.190287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-36f035ec-4a05-4f8b-8191-de0d5c93b25a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:28.191545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-36f035ec-4a05-4f8b-8191-de0d5c93b25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:28.229644 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b65fdf-3954-4e6b-a03d-c2941e933dcf None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.264155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250a7d5-e0c5-4271-89a2-547ad837da2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.264745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250a7d5-e0c5-4271-89a2-547ad837da2a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.265022 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250a7d5-e0c5-4271-89a2-547ad837da2a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.265285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250a7d5-e0c5-4271-89a2-547ad837da2a None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.265931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250a7d5-e0c5-4271-89a2-547ad837da2a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:28.266227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250a7d5-e0c5-4271-89a2-547ad837da2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.267569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250a7d5-e0c5-4271-89a2-547ad837da2a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.270243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 397/813] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:28.271039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f035ec-4a05-4f8b-8191-de0d5c93b25a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.277262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 417/814] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Aug 8 16:41:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:28.278061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250a7d5-e0c5-4271-89a2-547ad837da2a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.306736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:28.347682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:28.378866 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f035ec-4a05-4f8b-8191-de0d5c93b25a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.385618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250a7d5-e0c5-4271-89a2-547ad837da2a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.415440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c19df3-9c72-4b92-b2f5-3fe1d8c91039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.416155 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c19df3-9c72-4b92-b2f5-3fe1d8c91039 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.416499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c19df3-9c72-4b92-b2f5-3fe1d8c91039 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.416835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c19df3-9c72-4b92-b2f5-3fe1d8c91039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.423676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5bceb-17cb-4d05-a53d-659b27b454c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.424486 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5bceb-17cb-4d05-a53d-659b27b454c0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.424761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5bceb-17cb-4d05-a53d-659b27b454c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.425313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5bceb-17cb-4d05-a53d-659b27b454c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.427283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c19df3-9c72-4b92-b2f5-3fe1d8c91039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.429324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c19df3-9c72-4b92-b2f5-3fe1d8c91039 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.432674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5bceb-17cb-4d05-a53d-659b27b454c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.434136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5bceb-17cb-4d05-a53d-659b27b454c0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-d8c19df3-9c72-4b92-b2f5-3fe1d8c91039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:28.444178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:28.445134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:28.446932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:28.446932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:28.446932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:28.446932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 398/815] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:28.446932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c19df3-9c72-4b92-b2f5-3fe1d8c91039 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-ecf5bceb-17cb-4d05-a53d-659b27b454c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:28.450061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:28.450882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:28.451746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:28.451746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:28.451746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:28.451746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:28.451746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:28.451746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:28.451746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:28.451746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:28.451746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 418/816] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:28.453840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5bceb-17cb-4d05-a53d-659b27b454c0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.548489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:28.554757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c19df3-9c72-4b92-b2f5-3fe1d8c91039 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.561185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5bceb-17cb-4d05-a53d-659b27b454c0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.579095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:28.592825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a08335-7559-4814-ad1e-688b5a9fc81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.593447 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a08335-7559-4814-ad1e-688b5a9fc81b None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.593731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a08335-7559-4814-ad1e-688b5a9fc81b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.594224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a08335-7559-4814-ad1e-688b5a9fc81b None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.594879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a08335-7559-4814-ad1e-688b5a9fc81b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.596246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a08335-7559-4814-ad1e-688b5a9fc81b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.603890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01c069-64cc-49e3-a818-babf9554895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.604603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01c069-64cc-49e3-a818-babf9554895f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.604963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01c069-64cc-49e3-a818-babf9554895f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.605303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01c069-64cc-49e3-a818-babf9554895f None admin] PATH_INFO: `/v3/services/network` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.605982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01c069-64cc-49e3-a818-babf9554895f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.607403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01c069-64cc-49e3-a818-babf9554895f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-4f01c069-64cc-49e3-a818-babf9554895f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:28.617458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:28.622413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:28.623241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 419/817] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:28 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:28.624037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01c069-64cc-49e3-a818-babf9554895f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.705814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 399/818] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:41:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:28.705814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a08335-7559-4814-ad1e-688b5a9fc81b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.734579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01c069-64cc-49e3-a818-babf9554895f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.778646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb121ff-a73f-4cfb-b866-8a2c92d106d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.779266 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb121ff-a73f-4cfb-b866-8a2c92d106d4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.779546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb121ff-a73f-4cfb-b866-8a2c92d106d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.779820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb121ff-a73f-4cfb-b866-8a2c92d106d4 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.780476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb121ff-a73f-4cfb-b866-8a2c92d106d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:28.780778 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb121ff-a73f-4cfb-b866-8a2c92d106d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.782302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb121ff-a73f-4cfb-b866-8a2c92d106d4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.796301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 420/819] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:28.797177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb121ff-a73f-4cfb-b866-8a2c92d106d4 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.809061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:28.851441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a08335-7559-4814-ad1e-688b5a9fc81b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.900423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f59b4-86be-49ff-a7cd-c033fb58aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.900967 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f59b4-86be-49ff-a7cd-c033fb58aa9a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.901246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f59b4-86be-49ff-a7cd-c033fb58aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.901517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f59b4-86be-49ff-a7cd-c033fb58aa9a None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.903522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f59b4-86be-49ff-a7cd-c033fb58aa9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:28.903522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f59b4-86be-49ff-a7cd-c033fb58aa9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.905580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f59b4-86be-49ff-a7cd-c033fb58aa9a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.919275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb121ff-a73f-4cfb-b866-8a2c92d106d4 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:28.925045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 400/820] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:28.925885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f59b4-86be-49ff-a7cd-c033fb58aa9a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:28.972494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:28.976794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580088-a24c-4f05-a3b9-4aa3212ba3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:28.977815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580088-a24c-4f05-a3b9-4aa3212ba3dd None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:28.978715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580088-a24c-4f05-a3b9-4aa3212ba3dd None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:28.979355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580088-a24c-4f05-a3b9-4aa3212ba3dd None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:28.980267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580088-a24c-4f05-a3b9-4aa3212ba3dd None admin] RBAC: Adding query filter params (name=network) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:28.980926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580088-a24c-4f05-a3b9-4aa3212ba3dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:28.982801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580088-a24c-4f05-a3b9-4aa3212ba3dd None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:28.989664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 421/821] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:28.990769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580088-a24c-4f05-a3b9-4aa3212ba3dd None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:29.042469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f59b4-86be-49ff-a7cd-c033fb58aa9a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:29.080844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04054665-422c-45d9-b03f-5332476af8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:29.081981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04054665-422c-45d9-b03f-5332476af8f3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:29.082429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04054665-422c-45d9-b03f-5332476af8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:29.082839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04054665-422c-45d9-b03f-5332476af8f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:29.090264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:29.094832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04054665-422c-45d9-b03f-5332476af8f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:29.096436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04054665-422c-45d9-b03f-5332476af8f3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-04054665-422c-45d9-b03f-5332476af8f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:29.112760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:29.113956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:29.115107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:29.122113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 401/822] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Aug 8 16:41:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:29.124221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04054665-422c-45d9-b03f-5332476af8f3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:29.132813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580088-a24c-4f05-a3b9-4aa3212ba3dd None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:29.169154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b34f5e-15bd-46df-b8c2-ccb7ca7e9d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:29.170020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b34f5e-15bd-46df-b8c2-ccb7ca7e9d70 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:29.170351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b34f5e-15bd-46df-b8c2-ccb7ca7e9d70 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:29.170719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b34f5e-15bd-46df-b8c2-ccb7ca7e9d70 None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/92130daa3c3a49c2adc597dc563b2973/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:29.173141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b34f5e-15bd-46df-b8c2-ccb7ca7e9d70 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=92130daa3c3a49c2adc597dc563b2973, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:29.174823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b34f5e-15bd-46df-b8c2-ccb7ca7e9d70 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:29.239298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:29.242290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04054665-422c-45d9-b03f-5332476af8f3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:29.260141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 422/823] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:41:28 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/92130daa3c3a49c2adc597dc563b2973/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:41:29.261692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b34f5e-15bd-46df-b8c2-ccb7ca7e9d70 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:29.281557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f5f96-601d-4f26-b9b8-2b9130679812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:29.282354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f5f96-601d-4f26-b9b8-2b9130679812 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:29.282869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f5f96-601d-4f26-b9b8-2b9130679812 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:29.283158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f5f96-601d-4f26-b9b8-2b9130679812 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:29.283954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f5f96-601d-4f26-b9b8-2b9130679812 None admin] RBAC: Adding query filter params (type=network) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:29.284414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f5f96-601d-4f26-b9b8-2b9130679812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:29.286383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f5f96-601d-4f26-b9b8-2b9130679812 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:29.304492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 402/824] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:29.316616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f5f96-601d-4f26-b9b8-2b9130679812 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:29.408616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b34f5e-15bd-46df-b8c2-ccb7ca7e9d70 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:29.446254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:29.450480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b6027-fe48-44b2-92e2-cd36f2717e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:29.451090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b6027-fe48-44b2-92e2-cd36f2717e79 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:29.451366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b6027-fe48-44b2-92e2-cd36f2717e79 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:29.451633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b6027-fe48-44b2-92e2-cd36f2717e79 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:29.452362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b6027-fe48-44b2-92e2-cd36f2717e79 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:29.452664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b6027-fe48-44b2-92e2-cd36f2717e79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:29.454133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b6027-fe48-44b2-92e2-cd36f2717e79 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:29.473879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 423/825] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Aug 8 16:41:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:29.474765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f5f96-601d-4f26-b9b8-2b9130679812 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:29.475107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7eda4-459f-412c-a6b8-d42ff6cea35f None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:29.475641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7eda4-459f-412c-a6b8-d42ff6cea35f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:29.475998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7eda4-459f-412c-a6b8-d42ff6cea35f None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:29.477553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 424/826] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:29.482127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7eda4-459f-412c-a6b8-d42ff6cea35f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:29.517757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51e7df-ac33-4432-933d-a2695673026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:29.518449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51e7df-ac33-4432-933d-a2695673026c None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:29.518725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51e7df-ac33-4432-933d-a2695673026c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:29.518994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51e7df-ac33-4432-933d-a2695673026c None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:29.519651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51e7df-ac33-4432-933d-a2695673026c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:29.520999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51e7df-ac33-4432-933d-a2695673026c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:29.605841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 403/827] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:41:29 2022] POST /identity/v3/endpoints => generated 337 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:41:29.607426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd774646-2036-4db5-9c01-79cded78f2cb None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:29.608206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd774646-2036-4db5-9c01-79cded78f2cb None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:29.608687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd774646-2036-4db5-9c01-79cded78f2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:29.614842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7eda4-459f-412c-a6b8-d42ff6cea35f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:29.634868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-dd774646-2036-4db5-9c01-79cded78f2cb None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:29.637400 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-dd774646-2036-4db5-9c01-79cded78f2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:29.658083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:29.662227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5a652-c3ae-461c-9f4d-40a6622f1484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:29.662887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5a652-c3ae-461c-9f4d-40a6622f1484 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:29.663218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5a652-c3ae-461c-9f4d-40a6622f1484 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:29.663490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5a652-c3ae-461c-9f4d-40a6622f1484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:29.726397 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 404/828] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:29.750949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:29.755977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd774646-2036-4db5-9c01-79cded78f2cb None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:29.768500 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5a652-c3ae-461c-9f4d-40a6622f1484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:29.770619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5a652-c3ae-461c-9f4d-40a6622f1484 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:29.881513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd774646-2036-4db5-9c01-79cded78f2cb None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:29.901393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 425/829] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:29 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:41:29.910894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5a652-c3ae-461c-9f4d-40a6622f1484 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:29.921680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f60253-d32f-449f-a507-ee9722848d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:29.922708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f60253-d32f-449f-a507-ee9722848d20 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:29.923189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f60253-d32f-449f-a507-ee9722848d20 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:29.923660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f60253-d32f-449f-a507-ee9722848d20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:29.935169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f60253-d32f-449f-a507-ee9722848d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-67f60253-d32f-449f-a507-ee9722848d20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:29.941073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:29.942043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:41:29.943088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:29.945322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 405/830] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:41:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:29.951563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f60253-d32f-449f-a507-ee9722848d20 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:29.969258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:30.016216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:30.035598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5a652-c3ae-461c-9f4d-40a6622f1484 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.072525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f60253-d32f-449f-a507-ee9722848d20 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.077552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0133a-437d-4b53-a780-12c9eb9b7f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.078213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0133a-437d-4b53-a780-12c9eb9b7f5b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.078488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0133a-437d-4b53-a780-12c9eb9b7f5b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.078754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0133a-437d-4b53-a780-12c9eb9b7f5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.089875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0133a-437d-4b53-a780-12c9eb9b7f5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.091322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0133a-437d-4b53-a780-12c9eb9b7f5b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-ecc0133a-437d-4b53-a780-12c9eb9b7f5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:30.109300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:30.110320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:30.111317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:30.111317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:30.111317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:30.111317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 426/831] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:30.115707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd4f4b-5a0a-48d4-bf3d-28f3d239eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.116711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd4f4b-5a0a-48d4-bf3d-28f3d239eb71 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.117020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0133a-437d-4b53-a780-12c9eb9b7f5b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:30.117427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd4f4b-5a0a-48d4-bf3d-28f3d239eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.117954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd4f4b-5a0a-48d4-bf3d-28f3d239eb71 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.118781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd4f4b-5a0a-48d4-bf3d-28f3d239eb71 None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:30.119239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd4f4b-5a0a-48d4-bf3d-28f3d239eb71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.120996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd4f4b-5a0a-48d4-bf3d-28f3d239eb71 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.130462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 406/832] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:41:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:30.136622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd4f4b-5a0a-48d4-bf3d-28f3d239eb71 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:30.213322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:30.219872 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0133a-437d-4b53-a780-12c9eb9b7f5b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.244894 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd4f4b-5a0a-48d4-bf3d-28f3d239eb71 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.262005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d4445-a62c-4257-b399-e8dd7579c641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.262667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d4445-a62c-4257-b399-e8dd7579c641 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.262947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d4445-a62c-4257-b399-e8dd7579c641 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.263216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d4445-a62c-4257-b399-e8dd7579c641 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.263855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d4445-a62c-4257-b399-e8dd7579c641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:30.264155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d4445-a62c-4257-b399-e8dd7579c641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.265501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d4445-a62c-4257-b399-e8dd7579c641 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.280309 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ce3c9-d3f8-4c17-bbee-77fd18ce1277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.280870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ce3c9-d3f8-4c17-bbee-77fd18ce1277 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.281148 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ce3c9-d3f8-4c17-bbee-77fd18ce1277 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.281419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ce3c9-d3f8-4c17-bbee-77fd18ce1277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.286088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:30.297712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 427/833] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:30.298702 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d4445-a62c-4257-b399-e8dd7579c641 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:30.300270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ce3c9-d3f8-4c17-bbee-77fd18ce1277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.301763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ce3c9-d3f8-4c17-bbee-77fd18ce1277 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-f65ce3c9-d3f8-4c17-bbee-77fd18ce1277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:30.317564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:30.318757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:30.320215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:30.320215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:30.320215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:30.320215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 407/834] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:30.325921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ce3c9-d3f8-4c17-bbee-77fd18ce1277 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:30.426705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d4445-a62c-4257-b399-e8dd7579c641 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.443433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ce3c9-d3f8-4c17-bbee-77fd18ce1277 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.468897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:30.476364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b51640-b422-4f2a-a1c6-6ffa3e7727cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.477425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b51640-b422-4f2a-a1c6-6ffa3e7727cc None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.478036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b51640-b422-4f2a-a1c6-6ffa3e7727cc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.480455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b51640-b422-4f2a-a1c6-6ffa3e7727cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.480556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997aa28-5ce3-4ef8-9dea-cf7a57bd780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.480556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997aa28-5ce3-4ef8-9dea-cf7a57bd780b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.480556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997aa28-5ce3-4ef8-9dea-cf7a57bd780b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.480806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997aa28-5ce3-4ef8-9dea-cf7a57bd780b None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.481491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997aa28-5ce3-4ef8-9dea-cf7a57bd780b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:30.481812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997aa28-5ce3-4ef8-9dea-cf7a57bd780b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.483198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997aa28-5ce3-4ef8-9dea-cf7a57bd780b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.487971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b51640-b422-4f2a-a1c6-6ffa3e7727cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.489642 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b51640-b422-4f2a-a1c6-6ffa3e7727cc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.500659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 408/835] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:30.501550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997aa28-5ce3-4ef8-9dea-cf7a57bd780b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-40b51640-b422-4f2a-a1c6-6ffa3e7727cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:30.506007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:30.507437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:30.508290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:30.508290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:30.508290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:30.508290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:30.508290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:30.508290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:30.508290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:30.508290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:30.508290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 428/836] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:30.514541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b51640-b422-4f2a-a1c6-6ffa3e7727cc None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:30.606428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:30.633343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997aa28-5ce3-4ef8-9dea-cf7a57bd780b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.662599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b51640-b422-4f2a-a1c6-6ffa3e7727cc None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.671919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5f574-41aa-422e-b322-1bb489436ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.673222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5f574-41aa-422e-b322-1bb489436ff9 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.673499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5f574-41aa-422e-b322-1bb489436ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.673767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5f574-41aa-422e-b322-1bb489436ff9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.684479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5f574-41aa-422e-b322-1bb489436ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.685944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5f574-41aa-422e-b322-1bb489436ff9 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-18f5f574-41aa-422e-b322-1bb489436ff9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:30.701486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:30.702631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:30.703543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:30.705332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca2ce9-e017-4a4b-8a95-7cf56b9309bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.706253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca2ce9-e017-4a4b-8a95-7cf56b9309bb None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.706722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca2ce9-e017-4a4b-8a95-7cf56b9309bb None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.707373 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 409/837] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:41:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:30.707678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca2ce9-e017-4a4b-8a95-7cf56b9309bb None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.708239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5f574-41aa-422e-b322-1bb489436ff9 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:30.708936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca2ce9-e017-4a4b-8a95-7cf56b9309bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:30.709459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca2ce9-e017-4a4b-8a95-7cf56b9309bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.711190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca2ce9-e017-4a4b-8a95-7cf56b9309bb None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.729749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 429/838] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:30.730959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca2ce9-e017-4a4b-8a95-7cf56b9309bb None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:30.766083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:30.836235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca2ce9-e017-4a4b-8a95-7cf56b9309bb None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.840884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5f574-41aa-422e-b322-1bb489436ff9 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:30.856356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:30.874607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca857a-5db5-4f82-96dc-74839dafe93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.875510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca857a-5db5-4f82-96dc-74839dafe93f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.875982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca857a-5db5-4f82-96dc-74839dafe93f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.876461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca857a-5db5-4f82-96dc-74839dafe93f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.877408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca857a-5db5-4f82-96dc-74839dafe93f None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=54b05cbc41b844d8932ba4c15d960289) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:30.877963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca857a-5db5-4f82-96dc-74839dafe93f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.879648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca857a-5db5-4f82-96dc-74839dafe93f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.884643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db5039-415a-4afe-9dd4-ee02ec86f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:30.885273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db5039-415a-4afe-9dd4-ee02ec86f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.885610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db5039-415a-4afe-9dd4-ee02ec86f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.885973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db5039-415a-4afe-9dd4-ee02ec86f0f2 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.886717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db5039-415a-4afe-9dd4-ee02ec86f0f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:30.887023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db5039-415a-4afe-9dd4-ee02ec86f0f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:30.893844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db5039-415a-4afe-9dd4-ee02ec86f0f2 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:30.894327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 430/839] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:30 2022] GET /identity/v3/role_assignments?user.id=54b05cbc41b844d8932ba4c15d960289&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:30.910248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 410/840] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Aug 8 16:41:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:30.916582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca857a-5db5-4f82-96dc-74839dafe93f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:30.948328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f524514-9d0b-47fa-b5c5-e67d86f146bc None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.948851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f524514-9d0b-47fa-b5c5-e67d86f146bc None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.949266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f524514-9d0b-47fa-b5c5-e67d86f146bc None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.951271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 411/841] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:30.956940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e08ca-1b7d-40ee-bcf7-63d2e2e876f0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:30.957432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e08ca-1b7d-40ee-bcf7-63d2e2e876f0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:30.957879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e08ca-1b7d-40ee-bcf7-63d2e2e876f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:30.978476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e90e08ca-1b7d-40ee-bcf7-63d2e2e876f0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:30.979982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e90e08ca-1b7d-40ee-bcf7-63d2e2e876f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:31.051733 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca857a-5db5-4f82-96dc-74839dafe93f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:31.073832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 412/842] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:31.094775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e08ca-1b7d-40ee-bcf7-63d2e2e876f0 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:31.105029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7289b0-0c35-4b0a-95d8-c3147b4b0354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:31.105803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7289b0-0c35-4b0a-95d8-c3147b4b0354 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:31.106095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7289b0-0c35-4b0a-95d8-c3147b4b0354 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:31.106358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7289b0-0c35-4b0a-95d8-c3147b4b0354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:31.137637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:31.170139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:31.190746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7289b0-0c35-4b0a-95d8-c3147b4b0354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:31.192338 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7289b0-0c35-4b0a-95d8-c3147b4b0354 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:31.224719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e08ca-1b7d-40ee-bcf7-63d2e2e876f0 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:31.275628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310197a-0b3d-4a19-8da9-2f8e2b389e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:31.276846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310197a-0b3d-4a19-8da9-2f8e2b389e9b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:31.277176 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310197a-0b3d-4a19-8da9-2f8e2b389e9b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:31.277448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310197a-0b3d-4a19-8da9-2f8e2b389e9b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:31.278303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310197a-0b3d-4a19-8da9-2f8e2b389e9b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:31.280770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310197a-0b3d-4a19-8da9-2f8e2b389e9b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-1310197a-0b3d-4a19-8da9-2f8e2b389e9b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:31.291940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:31.292965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:41:31.293780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:31.294533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 413/843] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:41:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:31.300279 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310197a-0b3d-4a19-8da9-2f8e2b389e9b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:31.328283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 431/844] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:41:31.329540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7289b0-0c35-4b0a-95d8-c3147b4b0354 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:31.421874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:31.428229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:31.429821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310197a-0b3d-4a19-8da9-2f8e2b389e9b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:31.452075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7289b0-0c35-4b0a-95d8-c3147b4b0354 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:31.472962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b972a-4316-4daa-9f34-fcdd3be53f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:31.474086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b972a-4316-4daa-9f34-fcdd3be53f9e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:31.474561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b972a-4316-4daa-9f34-fcdd3be53f9e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:31.475021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b972a-4316-4daa-9f34-fcdd3be53f9e None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:31.475865 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b972a-4316-4daa-9f34-fcdd3be53f9e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:31.476359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b972a-4316-4daa-9f34-fcdd3be53f9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:31.478053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b972a-4316-4daa-9f34-fcdd3be53f9e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:31.483639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 414/845] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:41:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:31.485727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63101410-dafb-4cf6-96cc-2fcbfa334684 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:31.486106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63101410-dafb-4cf6-96cc-2fcbfa334684 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:31.486446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63101410-dafb-4cf6-96cc-2fcbfa334684 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:31.487879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 415/846] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:31.489058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63101410-dafb-4cf6-96cc-2fcbfa334684 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:31.495808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0115fb1-b1f7-4f7a-be37-4749c7afcf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:31.496448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0115fb1-b1f7-4f7a-be37-4749c7afcf3c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:31.496727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0115fb1-b1f7-4f7a-be37-4749c7afcf3c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:31.496995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0115fb1-b1f7-4f7a-be37-4749c7afcf3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:31.510472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0115fb1-b1f7-4f7a-be37-4749c7afcf3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:31.511907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0115fb1-b1f7-4f7a-be37-4749c7afcf3c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-d0115fb1-b1f7-4f7a-be37-4749c7afcf3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:31.527982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:31.528993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:31.530377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:31.530377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:31.530377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:31.530377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 432/847] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:31.530377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59391a4f-731e-4117-aef5-bb826846ea1a None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:31.530668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59391a4f-731e-4117-aef5-bb826846ea1a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:31.530735 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59391a4f-731e-4117-aef5-bb826846ea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:31.549861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-59391a4f-731e-4117-aef5-bb826846ea1a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:31.551034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-59391a4f-731e-4117-aef5-bb826846ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:31.613147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63101410-dafb-4cf6-96cc-2fcbfa334684 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:31.627359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 433/848] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:31.628432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59391a4f-731e-4117-aef5-bb826846ea1a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:31.659679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ce1ef-424d-4817-9eca-0c1a05673f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:31.660381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ce1ef-424d-4817-9eca-0c1a05673f97 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:31.660658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ce1ef-424d-4817-9eca-0c1a05673f97 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:31.660925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ce1ef-424d-4817-9eca-0c1a05673f97 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:31.661544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ce1ef-424d-4817-9eca-0c1a05673f97 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:31.661871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ce1ef-424d-4817-9eca-0c1a05673f97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:31.663248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ce1ef-424d-4817-9eca-0c1a05673f97 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:31.668768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 416/849] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:41:31 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:31.670855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ce1ef-424d-4817-9eca-0c1a05673f97 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:31.705644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:31.708720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:31.753343 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59391a4f-731e-4117-aef5-bb826846ea1a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:31.791569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcde1e8-d1e8-443a-a18e-e6305740e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:31.792112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ce1ef-424d-4817-9eca-0c1a05673f97 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:31.792225 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde1e8-d1e8-443a-a18e-e6305740e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:31.792453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde1e8-d1e8-443a-a18e-e6305740e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:31.792802 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde1e8-d1e8-443a-a18e-e6305740e9a1 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:31.793534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcde1e8-d1e8-443a-a18e-e6305740e9a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:31.793858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcde1e8-d1e8-443a-a18e-e6305740e9a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:31.795269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcde1e8-d1e8-443a-a18e-e6305740e9a1 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:31.809912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 434/850] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:31.814852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcde1e8-d1e8-443a-a18e-e6305740e9a1 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:31.833706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2ff62-35c4-4bb5-935d-5839f93bf709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:31.834815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2ff62-35c4-4bb5-935d-5839f93bf709 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:31.835325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2ff62-35c4-4bb5-935d-5839f93bf709 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:31.835793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2ff62-35c4-4bb5-935d-5839f93bf709 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:31.851376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2ff62-35c4-4bb5-935d-5839f93bf709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-28a2ff62-35c4-4bb5-935d-5839f93bf709 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:31.856140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:31.857070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:31.857911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:31.861870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 417/851] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:31.863867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2ff62-35c4-4bb5-935d-5839f93bf709 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:31.959207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:31.967087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcde1e8-d1e8-443a-a18e-e6305740e9a1 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:31.978867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2ff62-35c4-4bb5-935d-5839f93bf709 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:32.003166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:32.017205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcade3-5d49-4c36-8485-cae615b3b07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:32.017942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcade3-5d49-4c36-8485-cae615b3b07c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:32.018290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcade3-5d49-4c36-8485-cae615b3b07c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:32.018674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcade3-5d49-4c36-8485-cae615b3b07c None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:32.019191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23506b28-7f7c-4fc2-b262-658658e0c4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:32.019456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcade3-5d49-4c36-8485-cae615b3b07c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:32.019841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcade3-5d49-4c36-8485-cae615b3b07c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:32.020477 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23506b28-7f7c-4fc2-b262-658658e0c4f3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:32.020941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23506b28-7f7c-4fc2-b262-658658e0c4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:32.021387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcade3-5d49-4c36-8485-cae615b3b07c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:32.021617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23506b28-7f7c-4fc2-b262-658658e0c4f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:32.031168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23506b28-7f7c-4fc2-b262-658658e0c4f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:32.033262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23506b28-7f7c-4fc2-b262-658658e0c4f3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:32.036358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 418/852] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:32.038352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcade3-5d49-4c36-8485-cae615b3b07c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-23506b28-7f7c-4fc2-b262-658658e0c4f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:32.049144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:32.050103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:32.050988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:32.050988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:32.050988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:32.050988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:32.050988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:32.050988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:32.050988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:32.050988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:32.054247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 435/853] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:32.059764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23506b28-7f7c-4fc2-b262-658658e0c4f3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:32.168809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcade3-5d49-4c36-8485-cae615b3b07c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:32.176919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23506b28-7f7c-4fc2-b262-658658e0c4f3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:32.207712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffaf58-b5c0-46fb-99d2-e7137089952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:32.208343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffaf58-b5c0-46fb-99d2-e7137089952b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:32.208622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffaf58-b5c0-46fb-99d2-e7137089952b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:32.208893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffaf58-b5c0-46fb-99d2-e7137089952b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:32.215947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178feb-36da-4a2a-837f-e17b87bd866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:32.216935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178feb-36da-4a2a-837f-e17b87bd866a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:32.217416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178feb-36da-4a2a-837f-e17b87bd866a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:32.217970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178feb-36da-4a2a-837f-e17b87bd866a None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:32.218849 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0178feb-36da-4a2a-837f-e17b87bd866a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:32.219341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0178feb-36da-4a2a-837f-e17b87bd866a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:32.219714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffaf58-b5c0-46fb-99d2-e7137089952b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:32.221159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffaf58-b5c0-46fb-99d2-e7137089952b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:32.221452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0178feb-36da-4a2a-837f-e17b87bd866a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:32.228959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b0ffaf58-b5c0-46fb-99d2-e7137089952b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:32.237091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:32.245227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:32.246147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:32.246147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:32.246147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:32.246147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 419/854] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:32.246412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 436/855] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:32.249648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffaf58-b5c0-46fb-99d2-e7137089952b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:32.250886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178feb-36da-4a2a-837f-e17b87bd866a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:32.299017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:32.364332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178feb-36da-4a2a-837f-e17b87bd866a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:32.377877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffaf58-b5c0-46fb-99d2-e7137089952b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:32.414826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a700a3f-f2f1-43f7-b530-fa33ef6b1b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:32.415253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700a3f-f2f1-43f7-b530-fa33ef6b1b19 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:32.415531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700a3f-f2f1-43f7-b530-fa33ef6b1b19 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:32.415799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a700a3f-f2f1-43f7-b530-fa33ef6b1b19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:32.416523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a700a3f-f2f1-43f7-b530-fa33ef6b1b19 None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=19cf8bdae18e46ac8217c021765a2a78) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:32.416829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a700a3f-f2f1-43f7-b530-fa33ef6b1b19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:32.417023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9545ef3-6123-4d50-bbde-4d181913d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:32.417871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9545ef3-6123-4d50-bbde-4d181913d34d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:32.418297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a700a3f-f2f1-43f7-b530-fa33ef6b1b19 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:32.418567 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9545ef3-6123-4d50-bbde-4d181913d34d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:32.419017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9545ef3-6123-4d50-bbde-4d181913d34d None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:32.420180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9545ef3-6123-4d50-bbde-4d181913d34d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:32.420673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9545ef3-6123-4d50-bbde-4d181913d34d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:32.423150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9545ef3-6123-4d50-bbde-4d181913d34d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:32.429823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 420/856] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:32 2022] GET /identity/v3/role_assignments?user.id=19cf8bdae18e46ac8217c021765a2a78&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:32.441691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 437/857] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:32.447582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a700a3f-f2f1-43f7-b530-fa33ef6b1b19 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:32.518697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:32.568511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a700a3f-f2f1-43f7-b530-fa33ef6b1b19 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:32.609688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29ea9b-deff-416c-b1af-ed2345519b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:32.610717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29ea9b-deff-416c-b1af-ed2345519b73 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:32.611194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29ea9b-deff-416c-b1af-ed2345519b73 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:32.611664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29ea9b-deff-416c-b1af-ed2345519b73 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:32.620604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29ea9b-deff-416c-b1af-ed2345519b73 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:32.622425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29ea9b-deff-416c-b1af-ed2345519b73 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-1d29ea9b-deff-416c-b1af-ed2345519b73 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:32.634095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:32.635036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 16:41:32.635947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:32.639777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 421/858] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:32.641123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9545ef3-6123-4d50-bbde-4d181913d34d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:32.746796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9545ef3-6123-4d50-bbde-4d181913d34d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:32.750087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:32.788116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55bdd4-1840-4adb-8fe6-6805c353a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:32.788580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55bdd4-1840-4adb-8fe6-6805c353a020 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:32.788857 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55bdd4-1840-4adb-8fe6-6805c353a020 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:32.789122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55bdd4-1840-4adb-8fe6-6805c353a020 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:32.789880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55bdd4-1840-4adb-8fe6-6805c353a020 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:32.790184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55bdd4-1840-4adb-8fe6-6805c353a020 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:32.791822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55bdd4-1840-4adb-8fe6-6805c353a020 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:32.811036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 438/859] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Aug 8 16:41:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:32.816557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29ea9b-deff-416c-b1af-ed2345519b73 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:32.926107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29ea9b-deff-416c-b1af-ed2345519b73 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:32.950172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:32.964052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8e4a4-90bf-4563-8258-55388e25ca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:32.964975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8e4a4-90bf-4563-8258-55388e25ca35 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:32.965495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8e4a4-90bf-4563-8258-55388e25ca35 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:32.966008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8e4a4-90bf-4563-8258-55388e25ca35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.038106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8e4a4-90bf-4563-8258-55388e25ca35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:33.039956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8e4a4-90bf-4563-8258-55388e25ca35 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:33.064237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21453b39-cda0-4d7a-95a3-d5587ceda549 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.064566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21453b39-cda0-4d7a-95a3-d5587ceda549 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.064789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21453b39-cda0-4d7a-95a3-d5587ceda549 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.066543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 439/860] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:33.071468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a34522-ea58-425e-9ea4-3876e06419f9 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.071746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a34522-ea58-425e-9ea4-3876e06419f9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.072015 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a34522-ea58-425e-9ea4-3876e06419f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.089397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-10a34522-ea58-425e-9ea4-3876e06419f9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:33.090640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-10a34522-ea58-425e-9ea4-3876e06419f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:33.145921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 422/861] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:32 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:41:33.152134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8e4a4-90bf-4563-8258-55388e25ca35 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:33.168497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 440/862] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:33.183247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:33.190803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a34522-ea58-425e-9ea4-3876e06419f9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:33.285979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8e4a4-90bf-4563-8258-55388e25ca35 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:33.287064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:33.300391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a34522-ea58-425e-9ea4-3876e06419f9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:33.323280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6257f-52c2-44de-b6fd-2d2dcc989ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:33.323989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6257f-52c2-44de-b6fd-2d2dcc989ee4 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.324268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6257f-52c2-44de-b6fd-2d2dcc989ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.324540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6257f-52c2-44de-b6fd-2d2dcc989ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.335959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6257f-52c2-44de-b6fd-2d2dcc989ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:33.337512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6257f-52c2-44de-b6fd-2d2dcc989ee4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:33.348913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f212064-d2ec-4724-8caa-2d4bb967f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:33.349749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f212064-d2ec-4724-8caa-2d4bb967f564 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.350809 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f212064-d2ec-4724-8caa-2d4bb967f564 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.351242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f212064-d2ec-4724-8caa-2d4bb967f564 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.351979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f212064-d2ec-4724-8caa-2d4bb967f564 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-c1e6257f-52c2-44de-b6fd-2d2dcc989ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:33.353629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:33.354735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:33.355851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:33.355851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:33.355851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:33.356246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f212064-d2ec-4724-8caa-2d4bb967f564 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:33.356651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 423/863] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:33.362758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6257f-52c2-44de-b6fd-2d2dcc989ee4 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-0f212064-d2ec-4724-8caa-2d4bb967f564 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:33.366239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:33.367146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:41:33.368004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:33.374247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 441/864] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Aug 8 16:41:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:33.377931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f212064-d2ec-4724-8caa-2d4bb967f564 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:33.467983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:33.502004 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6257f-52c2-44de-b6fd-2d2dcc989ee4 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:33.504444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f212064-d2ec-4724-8caa-2d4bb967f564 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:33.545771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e8769-9768-4ebb-98a7-56abab216330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:33.547169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e8769-9768-4ebb-98a7-56abab216330 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.547627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e8769-9768-4ebb-98a7-56abab216330 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.548073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e8769-9768-4ebb-98a7-56abab216330 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.548892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e8769-9768-4ebb-98a7-56abab216330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:33.549405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e8769-9768-4ebb-98a7-56abab216330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:33.550532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44c9b6-67af-4ae6-ba0c-fa954941df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:33.551155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44c9b6-67af-4ae6-ba0c-fa954941df47 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.551436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e8769-9768-4ebb-98a7-56abab216330 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:33.551539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44c9b6-67af-4ae6-ba0c-fa954941df47 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.551813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44c9b6-67af-4ae6-ba0c-fa954941df47 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.552493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44c9b6-67af-4ae6-ba0c-fa954941df47 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:33.552853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44c9b6-67af-4ae6-ba0c-fa954941df47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:33.554502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44c9b6-67af-4ae6-ba0c-fa954941df47 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:33.559780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 442/865] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:33.565245 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44c9b6-67af-4ae6-ba0c-fa954941df47 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:33.568403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 424/866] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:33.574778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e8769-9768-4ebb-98a7-56abab216330 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:33.594326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:33.683787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44c9b6-67af-4ae6-ba0c-fa954941df47 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:33.702373 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e8769-9768-4ebb-98a7-56abab216330 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:33.730605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7180a-379f-441f-aaa6-5b7177506a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:33.731701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7180a-379f-441f-aaa6-5b7177506a67 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.732515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7180a-379f-441f-aaa6-5b7177506a67 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.733100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7180a-379f-441f-aaa6-5b7177506a67 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.734006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7180a-379f-441f-aaa6-5b7177506a67 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:33.734701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7180a-379f-441f-aaa6-5b7177506a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:33.736902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7180a-379f-441f-aaa6-5b7177506a67 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:33.741344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f39413-69e4-41e9-9d2e-db4ee94a70ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:33.742130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f39413-69e4-41e9-9d2e-db4ee94a70ee None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.742497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f39413-69e4-41e9-9d2e-db4ee94a70ee None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.742856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f39413-69e4-41e9-9d2e-db4ee94a70ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.743664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 443/867] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:33.746054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:33.752804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f39413-69e4-41e9-9d2e-db4ee94a70ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:33.754813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f39413-69e4-41e9-9d2e-db4ee94a70ee None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:33.769373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67415f33-2cf1-423c-b003-af79c64996bc None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.769949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67415f33-2cf1-423c-b003-af79c64996bc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.769949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67415f33-2cf1-423c-b003-af79c64996bc None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.771568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 444/868] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-f2f39413-69e4-41e9-9d2e-db4ee94a70ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:33.774775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:33.775815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:33.776723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:33.776723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:33.776723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:33.776723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:33.776723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:33.776723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:33.776723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:33.776723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:33.776723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 425/869] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:33.779400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232fab2-88b8-4a7f-80a5-a91a2b706972 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.779693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232fab2-88b8-4a7f-80a5-a91a2b706972 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.779957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232fab2-88b8-4a7f-80a5-a91a2b706972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.782778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f39413-69e4-41e9-9d2e-db4ee94a70ee None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:33.802962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9232fab2-88b8-4a7f-80a5-a91a2b706972 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:33.804973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9232fab2-88b8-4a7f-80a5-a91a2b706972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:33.890182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 445/870] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:33.905377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f39413-69e4-41e9-9d2e-db4ee94a70ee None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:33.913901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232fab2-88b8-4a7f-80a5-a91a2b706972 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:33.931498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:33.956528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8f5ce-1ac1-4d2b-97fe-c6c82e5ce276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:33.958185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8f5ce-1ac1-4d2b-97fe-c6c82e5ce276 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:33.959334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8f5ce-1ac1-4d2b-97fe-c6c82e5ce276 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:33.960486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8f5ce-1ac1-4d2b-97fe-c6c82e5ce276 None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:33.962285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8f5ce-1ac1-4d2b-97fe-c6c82e5ce276 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:33.962849 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8f5ce-1ac1-4d2b-97fe-c6c82e5ce276 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:33.964645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8f5ce-1ac1-4d2b-97fe-c6c82e5ce276 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:33.977613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 426/871] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:33.983598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8f5ce-1ac1-4d2b-97fe-c6c82e5ce276 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:34.046069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232fab2-88b8-4a7f-80a5-a91a2b706972 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:34.067755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:34.092424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dd389-f6c4-41d5-8312-8d848831a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:34.093038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dd389-f6c4-41d5-8312-8d848831a2dc None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:34.093360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dd389-f6c4-41d5-8312-8d848831a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:34.093633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dd389-f6c4-41d5-8312-8d848831a2dc None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:34.094388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dd389-f6c4-41d5-8312-8d848831a2dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:34.096385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dd389-f6c4-41d5-8312-8d848831a2dc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:34.122661 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8f5ce-1ac1-4d2b-97fe-c6c82e5ce276 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:34.158979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302e6d0-e5d7-42a0-8ef3-33f3e54c8665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:34.159909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302e6d0-e5d7-42a0-8ef3-33f3e54c8665 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:34.160406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302e6d0-e5d7-42a0-8ef3-33f3e54c8665 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:34.160868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302e6d0-e5d7-42a0-8ef3-33f3e54c8665 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:34.161819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302e6d0-e5d7-42a0-8ef3-33f3e54c8665 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=92130daa3c3a49c2adc597dc563b2973) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:34.162443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302e6d0-e5d7-42a0-8ef3-33f3e54c8665 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:34.164674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302e6d0-e5d7-42a0-8ef3-33f3e54c8665 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:34.171356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 427/872] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:33 2022] GET /identity/v3/role_assignments?user.id=92130daa3c3a49c2adc597dc563b2973&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:34.195667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 446/873] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:41:33 2022] POST /identity/v3/services => generated 251 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:41:34.688311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2b735-90d6-424f-bb81-b311fec34c01 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:34.688824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2b735-90d6-424f-bb81-b311fec34c01 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:34.688824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2b735-90d6-424f-bb81-b311fec34c01 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:34.690470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 428/874] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:34.697212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16028b63-9294-4677-b2a1-62c54b66a6fa None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:34.697751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16028b63-9294-4677-b2a1-62c54b66a6fa None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:34.698253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16028b63-9294-4677-b2a1-62c54b66a6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:34.718633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-16028b63-9294-4677-b2a1-62c54b66a6fa None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:34.720167 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-16028b63-9294-4677-b2a1-62c54b66a6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:34.789928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:34.795458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 447/875] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:34 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:34.819633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2b735-90d6-424f-bb81-b311fec34c01 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:34.936110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2b735-90d6-424f-bb81-b311fec34c01 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:34.955084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:34.972816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bc6ab-8734-4233-9586-1f81bb5e2357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:34.973449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bc6ab-8734-4233-9586-1f81bb5e2357 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:34.973727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bc6ab-8734-4233-9586-1f81bb5e2357 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:34.974027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bc6ab-8734-4233-9586-1f81bb5e2357 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:34.984397 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bc6ab-8734-4233-9586-1f81bb5e2357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-1a8bc6ab-8734-4233-9586-1f81bb5e2357 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:34.989039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:34.989991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:34.991380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 429/876] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Aug 8 16:41:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:35.000337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16028b63-9294-4677-b2a1-62c54b66a6fa None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:35.103442 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16028b63-9294-4677-b2a1-62c54b66a6fa None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:35.137958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5288b-e459-4854-9968-1b431863982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:35.138557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5288b-e459-4854-9968-1b431863982d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:35.138832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5288b-e459-4854-9968-1b431863982d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:35.139098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5288b-e459-4854-9968-1b431863982d None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:35.139750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5288b-e459-4854-9968-1b431863982d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:35.140048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5288b-e459-4854-9968-1b431863982d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:35.141396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5288b-e459-4854-9968-1b431863982d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:35.150301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 448/877] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Aug 8 16:41:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:35.158856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bc6ab-8734-4233-9586-1f81bb5e2357 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:35.263179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bc6ab-8734-4233-9586-1f81bb5e2357 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:35.302832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ef828-9bd5-4e1b-b896-2350718f97c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:35.303497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ef828-9bd5-4e1b-b896-2350718f97c7 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:35.303777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ef828-9bd5-4e1b-b896-2350718f97c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:35.304046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ef828-9bd5-4e1b-b896-2350718f97c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:35.314441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ef828-9bd5-4e1b-b896-2350718f97c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:35.315897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ef828-9bd5-4e1b-b896-2350718f97c7 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-083ef828-9bd5-4e1b-b896-2350718f97c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:35.331429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:35.338664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:35.339528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:35.339528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:35.339528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:35.339528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 430/878] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:35.343009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5288b-e459-4854-9968-1b431863982d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:35.410510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:35.451037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5288b-e459-4854-9968-1b431863982d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:35.493136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3c559-e743-4d30-af41-0ab4fa2277d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:35.494210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3c559-e743-4d30-af41-0ab4fa2277d5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:35.494691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3c559-e743-4d30-af41-0ab4fa2277d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:35.495174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3c559-e743-4d30-af41-0ab4fa2277d5 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:35.496018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3c559-e743-4d30-af41-0ab4fa2277d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:35.496526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3c559-e743-4d30-af41-0ab4fa2277d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:35.498303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3c559-e743-4d30-af41-0ab4fa2277d5 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:35.512264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 449/879] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:35.517452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ef828-9bd5-4e1b-b896-2350718f97c7 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:35.558196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:35.628338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ef828-9bd5-4e1b-b896-2350718f97c7 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:35.663343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e533ab2-1db4-4084-ba62-1ddbe238f84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:35.663978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533ab2-1db4-4084-ba62-1ddbe238f84b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:35.664252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533ab2-1db4-4084-ba62-1ddbe238f84b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:35.664519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533ab2-1db4-4084-ba62-1ddbe238f84b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:35.672361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e533ab2-1db4-4084-ba62-1ddbe238f84b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:35.673817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e533ab2-1db4-4084-ba62-1ddbe238f84b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-4e533ab2-1db4-4084-ba62-1ddbe238f84b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:35.684789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:35.685696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:35.687265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 431/880] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:41:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:35.691581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3c559-e743-4d30-af41-0ab4fa2277d5 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:35.800137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3c559-e743-4d30-af41-0ab4fa2277d5 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:35.806329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04833920-a5b6-4b16-acea-30a8555e408e None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:35.806850 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04833920-a5b6-4b16-acea-30a8555e408e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:35.807023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04833920-a5b6-4b16-acea-30a8555e408e None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:35.808686 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 432/881] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:35.814986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7809c-1f78-4243-8b5b-bb0594703aa2 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:35.815385 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7809c-1f78-4243-8b5b-bb0594703aa2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:35.815754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7809c-1f78-4243-8b5b-bb0594703aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:35.825744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:35.835102 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e1e7809c-1f78-4243-8b5b-bb0594703aa2 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:35.836408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e1e7809c-1f78-4243-8b5b-bb0594703aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:35.839004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fce18-32cb-457e-8f42-ce6d8432d63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:35.839715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fce18-32cb-457e-8f42-ce6d8432d63a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:35.839997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fce18-32cb-457e-8f42-ce6d8432d63a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:35.840270 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fce18-32cb-457e-8f42-ce6d8432d63a None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:35.841001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fce18-32cb-457e-8f42-ce6d8432d63a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:35.841304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fce18-32cb-457e-8f42-ce6d8432d63a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:35.842756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fce18-32cb-457e-8f42-ce6d8432d63a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:35.863496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 450/882] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Aug 8 16:41:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:35.869157 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fce18-32cb-457e-8f42-ce6d8432d63a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:35.913892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 433/883] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:35 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:35.931305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7809c-1f78-4243-8b5b-bb0594703aa2 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:35.954071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:35.982337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fce18-32cb-457e-8f42-ce6d8432d63a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:36.020258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf669052-8b56-4141-ac92-567439587a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:36.021023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf669052-8b56-4141-ac92-567439587a2d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.021303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf669052-8b56-4141-ac92-567439587a2d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.021671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf669052-8b56-4141-ac92-567439587a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.034984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7809c-1f78-4243-8b5b-bb0594703aa2 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:36.076145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:36.077839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06907b81-4036-4e03-8248-2f3f54276301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:36.078218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06907b81-4036-4e03-8248-2f3f54276301 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.078491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06907b81-4036-4e03-8248-2f3f54276301 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.078758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06907b81-4036-4e03-8248-2f3f54276301 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.079411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06907b81-4036-4e03-8248-2f3f54276301 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:36.080775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06907b81-4036-4e03-8248-2f3f54276301 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:36.100812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf669052-8b56-4141-ac92-567439587a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:36.102596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf669052-8b56-4141-ac92-567439587a2d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:36.185288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 434/884] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:41:35 2022] POST /identity/v3/services => generated 269 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:36.222221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 451/885] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:35 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 16:41:36.229245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06907b81-4036-4e03-8248-2f3f54276301 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:36.269002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ee48a-15a1-48d5-85de-ca9109b01d35 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.269342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ee48a-15a1-48d5-85de-ca9109b01d35 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.269545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ee48a-15a1-48d5-85de-ca9109b01d35 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.273071 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 452/886] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:36.277588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b2fca-9f51-4a88-9ab8-16a30c645ce1 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.277902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b2fca-9f51-4a88-9ab8-16a30c645ce1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.278176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b2fca-9f51-4a88-9ab8-16a30c645ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.298433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c39b2fca-9f51-4a88-9ab8-16a30c645ce1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:36.299736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c39b2fca-9f51-4a88-9ab8-16a30c645ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:36.325274 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:36.352376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06907b81-4036-4e03-8248-2f3f54276301 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:36.392882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 453/887] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:36.394112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07a550-d8dc-4d37-9bd2-4169ce196bc9 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.394400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07a550-d8dc-4d37-9bd2-4169ce196bc9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.394655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07a550-d8dc-4d37-9bd2-4169ce196bc9 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.395833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d76ebd-cc94-4621-b29d-9a81a8e5746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:36.396094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 454/888] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:36.396463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d76ebd-cc94-4621-b29d-9a81a8e5746e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.397297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d76ebd-cc94-4621-b29d-9a81a8e5746e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.397564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d76ebd-cc94-4621-b29d-9a81a8e5746e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.401601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc859da-add0-43af-bce8-5014acccc8b6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.401922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc859da-add0-43af-bce8-5014acccc8b6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.402198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc859da-add0-43af-bce8-5014acccc8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.411117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d76ebd-cc94-4621-b29d-9a81a8e5746e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:36.414186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d76ebd-cc94-4621-b29d-9a81a8e5746e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:36.424176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ebc859da-add0-43af-bce8-5014acccc8b6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:36.425379 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ebc859da-add0-43af-bce8-5014acccc8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:36.429011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b0d76ebd-cc94-4621-b29d-9a81a8e5746e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:36.434308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:36.435246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:36.436171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:36.436171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:36.436171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:36.436171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 435/889] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:36.436171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d76ebd-cc94-4621-b29d-9a81a8e5746e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:36.498072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 455/890] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:36.507885 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc859da-add0-43af-bce8-5014acccc8b6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:36.544754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d76ebd-cc94-4621-b29d-9a81a8e5746e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:36.575946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:36.583764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecda626-f4f7-4416-a929-0c83a3355fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:36.584879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecda626-f4f7-4416-a929-0c83a3355fcc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.585157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecda626-f4f7-4416-a929-0c83a3355fcc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.585421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecda626-f4f7-4416-a929-0c83a3355fcc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.586050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecda626-f4f7-4416-a929-0c83a3355fcc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:36.589339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecda626-f4f7-4416-a929-0c83a3355fcc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-3ecda626-f4f7-4416-a929-0c83a3355fcc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:36.600423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:36.601414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:36.602272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 436/891] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:36.603118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecda626-f4f7-4416-a929-0c83a3355fcc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:36.624222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc859da-add0-43af-bce8-5014acccc8b6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:36.659148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d8ff9-7cab-42df-937a-ef63eb8e8138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:36.659788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8ff9-7cab-42df-937a-ef63eb8e8138 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.660025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8ff9-7cab-42df-937a-ef63eb8e8138 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.660293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8ff9-7cab-42df-937a-ef63eb8e8138 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.660933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d8ff9-7cab-42df-937a-ef63eb8e8138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:36.661346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d8ff9-7cab-42df-937a-ef63eb8e8138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:36.663166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d8ff9-7cab-42df-937a-ef63eb8e8138 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:36.672074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:36.679394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 456/892] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:36.680271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d8ff9-7cab-42df-937a-ef63eb8e8138 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:36.725757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecda626-f4f7-4416-a929-0c83a3355fcc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:36.761316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4113c4c-9a06-4ae1-bac9-06a3e0c2f92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:36.761980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4113c4c-9a06-4ae1-bac9-06a3e0c2f92b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.762263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4113c4c-9a06-4ae1-bac9-06a3e0c2f92b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.762533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4113c4c-9a06-4ae1-bac9-06a3e0c2f92b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.763111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4113c4c-9a06-4ae1-bac9-06a3e0c2f92b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:36.764468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4113c4c-9a06-4ae1-bac9-06a3e0c2f92b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-c4113c4c-9a06-4ae1-bac9-06a3e0c2f92b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:36.773660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:36.774657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:36.775484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 437/893] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:41:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:36.776287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4113c4c-9a06-4ae1-bac9-06a3e0c2f92b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:36.793861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d8ff9-7cab-42df-937a-ef63eb8e8138 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:36.827637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078267b-0cc2-4444-a300-35f048e2ae74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:36.828262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078267b-0cc2-4444-a300-35f048e2ae74 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.828493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078267b-0cc2-4444-a300-35f048e2ae74 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.828760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078267b-0cc2-4444-a300-35f048e2ae74 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.829380 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078267b-0cc2-4444-a300-35f048e2ae74 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:36.829677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078267b-0cc2-4444-a300-35f048e2ae74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:36.831059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078267b-0cc2-4444-a300-35f048e2ae74 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:36.836244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 457/894] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:36.837010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078267b-0cc2-4444-a300-35f048e2ae74 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:36.841317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:36.892158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4113c4c-9a06-4ae1-bac9-06a3e0c2f92b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:36.899697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:36.928242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacacef9-54c0-465e-a3e6-87891d559a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:36.928873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacacef9-54c0-465e-a3e6-87891d559a0c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.929173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacacef9-54c0-465e-a3e6-87891d559a0c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.929417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacacef9-54c0-465e-a3e6-87891d559a0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.936796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacacef9-54c0-465e-a3e6-87891d559a0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:36.938272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacacef9-54c0-465e-a3e6-87891d559a0c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:36.939351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078267b-0cc2-4444-a300-35f048e2ae74 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-cacacef9-54c0-465e-a3e6-87891d559a0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:36.951535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:36.952644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 438/895] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:36.953543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacacef9-54c0-465e-a3e6-87891d559a0c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:36.981229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d52ff-cd98-44fe-a351-14981ed923bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:36.981867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d52ff-cd98-44fe-a351-14981ed923bc None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:36.982153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d52ff-cd98-44fe-a351-14981ed923bc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:36.982424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d52ff-cd98-44fe-a351-14981ed923bc None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:36.983054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d52ff-cd98-44fe-a351-14981ed923bc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:36.983355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d52ff-cd98-44fe-a351-14981ed923bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:36.984715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d52ff-cd98-44fe-a351-14981ed923bc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:36.990024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 458/896] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:41:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:36.991328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d52ff-cd98-44fe-a351-14981ed923bc None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:37.065587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacacef9-54c0-465e-a3e6-87891d559a0c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:37.082809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:37.097043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d52ff-cd98-44fe-a351-14981ed923bc None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:37.102318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a5b01-8b16-4189-bc6b-9d328e6e17d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:37.103181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a5b01-8b16-4189-bc6b-9d328e6e17d3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:37.103787 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a5b01-8b16-4189-bc6b-9d328e6e17d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:37.104301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a5b01-8b16-4189-bc6b-9d328e6e17d3 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:37.105035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a5b01-8b16-4189-bc6b-9d328e6e17d3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:37.105340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a5b01-8b16-4189-bc6b-9d328e6e17d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:37.106837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a5b01-8b16-4189-bc6b-9d328e6e17d3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:37.111604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 439/897] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:37.112392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a5b01-8b16-4189-bc6b-9d328e6e17d3 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:37.139593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:37.157768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c9ba1-d94d-492b-bb27-4f1a79b4bb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:37.157768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c9ba1-d94d-492b-bb27-4f1a79b4bb63 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:37.157768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c9ba1-d94d-492b-bb27-4f1a79b4bb63 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:37.157768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c9ba1-d94d-492b-bb27-4f1a79b4bb63 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:37.158188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c9ba1-d94d-492b-bb27-4f1a79b4bb63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:37.158188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c9ba1-d94d-492b-bb27-4f1a79b4bb63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:37.158188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c9ba1-d94d-492b-bb27-4f1a79b4bb63 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:37.174659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 459/898] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:37.182141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c9ba1-d94d-492b-bb27-4f1a79b4bb63 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:37.233202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a5b01-8b16-4189-bc6b-9d328e6e17d3 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:37.276611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c6473-3088-4331-a5dd-32a98d418fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:37.277228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c6473-3088-4331-a5dd-32a98d418fc5 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:37.277511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c6473-3088-4331-a5dd-32a98d418fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:37.277780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c6473-3088-4331-a5dd-32a98d418fc5 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:37.278438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c6473-3088-4331-a5dd-32a98d418fc5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:37.278737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c6473-3088-4331-a5dd-32a98d418fc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:37.280209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c6473-3088-4331-a5dd-32a98d418fc5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:37.287018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 440/899] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:41:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:37.293059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c6473-3088-4331-a5dd-32a98d418fc5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:37.325489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c9ba1-d94d-492b-bb27-4f1a79b4bb63 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:37.354162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:37.376520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f575f-0768-4ad4-bdf2-eaf53663fa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:37.377434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f575f-0768-4ad4-bdf2-eaf53663fa52 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:37.377728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f575f-0768-4ad4-bdf2-eaf53663fa52 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:37.378030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f575f-0768-4ad4-bdf2-eaf53663fa52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:37.379032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f575f-0768-4ad4-bdf2-eaf53663fa52 None admin] RBAC: Adding query filter params (role.id=3455a742ed0a4787a4e60d7f104a586a, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=19cf8bdae18e46ac8217c021765a2a78) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:37.379339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f575f-0768-4ad4-bdf2-eaf53663fa52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:37.380779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f575f-0768-4ad4-bdf2-eaf53663fa52 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:37.389148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 460/900] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:37 2022] GET /identity/v3/role_assignments?user.id=19cf8bdae18e46ac8217c021765a2a78&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=3455a742ed0a4787a4e60d7f104a586a => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:37.425142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c6473-3088-4331-a5dd-32a98d418fc5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:37.465179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797ba06-b40a-4440-911b-45da6fb0a03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:37.465819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797ba06-b40a-4440-911b-45da6fb0a03f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:37.466134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797ba06-b40a-4440-911b-45da6fb0a03f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:37.466404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797ba06-b40a-4440-911b-45da6fb0a03f None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:37.467196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797ba06-b40a-4440-911b-45da6fb0a03f None admin] RBAC: Adding query filter params (interface=public, service_id=de34c8f8677d4af989fd0c8a3935ce7c, region_id=RegionOne) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:37.467503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797ba06-b40a-4440-911b-45da6fb0a03f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:37.468964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797ba06-b40a-4440-911b-45da6fb0a03f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:37.482124 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 441/901] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Aug 8 16:41:37 2022] GET /identity/v3/endpoints?service_id=de34c8f8677d4af989fd0c8a3935ce7c&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:37.483417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f575f-0768-4ad4-bdf2-eaf53663fa52 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:37.549013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:37.604271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f575f-0768-4ad4-bdf2-eaf53663fa52 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:37.640944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f96df-ae13-4fe6-b2c3-2b91502a8a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:37.641552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f96df-ae13-4fe6-b2c3-2b91502a8a4b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:37.641957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f96df-ae13-4fe6-b2c3-2b91502a8a4b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:37.642230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f96df-ae13-4fe6-b2c3-2b91502a8a4b None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:37.642835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f96df-ae13-4fe6-b2c3-2b91502a8a4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:37.644194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f96df-ae13-4fe6-b2c3-2b91502a8a4b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:37.650836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 461/902] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Aug 8 16:41:37 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:41:38.271412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce9355-dcea-4529-89ca-5754f4ca25f2 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:38.271970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce9355-dcea-4529-89ca-5754f4ca25f2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:38.271970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce9355-dcea-4529-89ca-5754f4ca25f2 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:38.273598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 442/903] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:38.279268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27958c-9870-446c-a536-2d2b3f9d608b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:38.279779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27958c-9870-446c-a536-2d2b3f9d608b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:38.279779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27958c-9870-446c-a536-2d2b3f9d608b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:38.299891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2e27958c-9870-446c-a536-2d2b3f9d608b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:38.301086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2e27958c-9870-446c-a536-2d2b3f9d608b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:38.375169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 462/904] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:38.396901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce9355-dcea-4529-89ca-5754f4ca25f2 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:38.504633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce9355-dcea-4529-89ca-5754f4ca25f2 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:38.514131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:38.541345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4f733-861d-4273-83da-711d0992400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:38.542007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4f733-861d-4273-83da-711d0992400a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:38.542340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4f733-861d-4273-83da-711d0992400a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:38.542611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4f733-861d-4273-83da-711d0992400a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:38.543265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4f733-861d-4273-83da-711d0992400a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:38.544749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4f733-861d-4273-83da-711d0992400a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-69f4f733-861d-4273-83da-711d0992400a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:38.554618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:38.555544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:38.556601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 443/905] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Aug 8 16:41:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:38.563797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27958c-9870-446c-a536-2d2b3f9d608b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:38.623810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:38.678771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27958c-9870-446c-a536-2d2b3f9d608b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:38.713358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ff0a3-b354-41b5-9081-e4e0b865850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:38.713998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ff0a3-b354-41b5-9081-e4e0b865850f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:38.714274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ff0a3-b354-41b5-9081-e4e0b865850f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:38.714535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ff0a3-b354-41b5-9081-e4e0b865850f None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:38.715165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ff0a3-b354-41b5-9081-e4e0b865850f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:38.715463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ff0a3-b354-41b5-9081-e4e0b865850f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:38.716813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ff0a3-b354-41b5-9081-e4e0b865850f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:38.721827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 463/906] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:38.727877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4f733-861d-4273-83da-711d0992400a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:38.840748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4f733-861d-4273-83da-711d0992400a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:38.877025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52040b56-f7ad-4041-b401-ab2138daf4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:38.877639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52040b56-f7ad-4041-b401-ab2138daf4ae None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:38.877944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52040b56-f7ad-4041-b401-ab2138daf4ae None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:38.878213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52040b56-f7ad-4041-b401-ab2138daf4ae None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:38.878842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52040b56-f7ad-4041-b401-ab2138daf4ae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:38.879197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52040b56-f7ad-4041-b401-ab2138daf4ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:38.880568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52040b56-f7ad-4041-b401-ab2138daf4ae None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:38.886633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 444/907] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:38.892127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ff0a3-b354-41b5-9081-e4e0b865850f None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:39.000340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ff0a3-b354-41b5-9081-e4e0b865850f None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:39.011196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:39.035028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1badaf-15fc-4335-b213-4fbe52115acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:39.035638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1badaf-15fc-4335-b213-4fbe52115acc None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.035912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1badaf-15fc-4335-b213-4fbe52115acc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.036182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1badaf-15fc-4335-b213-4fbe52115acc None admin] PATH_INFO: `/v3/endpoints` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.036875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1badaf-15fc-4335-b213-4fbe52115acc None admin] RBAC: Adding query filter params (interface=public, service_id=4655379cd5004b4caf8c8a3eaa2c131f, region_id=RegionOne) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:39.037722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1badaf-15fc-4335-b213-4fbe52115acc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:39.039112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1badaf-15fc-4335-b213-4fbe52115acc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:39.044668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 464/908] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Aug 8 16:41:38 2022] GET /identity/v3/endpoints?service_id=4655379cd5004b4caf8c8a3eaa2c131f&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:39.049902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52040b56-f7ad-4041-b401-ab2138daf4ae None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:39.087152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:39.154346 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52040b56-f7ad-4041-b401-ab2138daf4ae None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:39.190573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43666c20-31ba-4d8c-8f28-acdaf95af198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:39.191228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43666c20-31ba-4d8c-8f28-acdaf95af198 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.191503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43666c20-31ba-4d8c-8f28-acdaf95af198 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.191868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43666c20-31ba-4d8c-8f28-acdaf95af198 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.192470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43666c20-31ba-4d8c-8f28-acdaf95af198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:39.194502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43666c20-31ba-4d8c-8f28-acdaf95af198 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:39.205967 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 445/909] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Aug 8 16:41:39 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:41:39.248888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102a425-0a33-49cc-b8bf-1c5732b2348d None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.249211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102a425-0a33-49cc-b8bf-1c5732b2348d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.249438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102a425-0a33-49cc-b8bf-1c5732b2348d None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.251187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 465/910] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:39.257837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c87bce-0c06-4aaf-ab8f-4ac0facd4de1 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.258128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c87bce-0c06-4aaf-ab8f-4ac0facd4de1 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.258364 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c87bce-0c06-4aaf-ab8f-4ac0facd4de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.277945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-04c87bce-0c06-4aaf-ab8f-4ac0facd4de1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:39.279169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-04c87bce-0c06-4aaf-ab8f-4ac0facd4de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:39.356500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 446/911] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:39.379856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102a425-0a33-49cc-b8bf-1c5732b2348d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:39.487627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102a425-0a33-49cc-b8bf-1c5732b2348d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:39.507569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db55e88-2b2a-4c13-a3d6-ca5eb8c205a7 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.507872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db55e88-2b2a-4c13-a3d6-ca5eb8c205a7 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.508107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db55e88-2b2a-4c13-a3d6-ca5eb8c205a7 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.513281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 447/912] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:39.514899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa24b36-d319-47bf-bd01-eaa7d621c924 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.515180 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa24b36-d319-47bf-bd01-eaa7d621c924 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.515449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa24b36-d319-47bf-bd01-eaa7d621c924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.523940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71972abe-760b-4f48-a967-2dc1e5186996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:39.524554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972abe-760b-4f48-a967-2dc1e5186996 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.524832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972abe-760b-4f48-a967-2dc1e5186996 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.525100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972abe-760b-4f48-a967-2dc1e5186996 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.525744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71972abe-760b-4f48-a967-2dc1e5186996 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:39.527147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71972abe-760b-4f48-a967-2dc1e5186996 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:39.536399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-bfa24b36-d319-47bf-bd01-eaa7d621c924 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:39.537735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-bfa24b36-d319-47bf-bd01-eaa7d621c924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:39.562616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:39.602932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 466/913] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:41:39 2022] POST /identity/v3/services => generated 241 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:39.625439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 448/914] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:39.647218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71972abe-760b-4f48-a967-2dc1e5186996 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:39.686617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:39.763322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44974-911a-41bb-9925-1b3d884bf0aa None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.763505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44974-911a-41bb-9925-1b3d884bf0aa None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.763784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44974-911a-41bb-9925-1b3d884bf0aa None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.765466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 449/915] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:41:39.766339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71972abe-760b-4f48-a967-2dc1e5186996 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:39.772090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9193cfd-132b-4639-82ef-67b3c8f8a378 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.772386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9193cfd-132b-4639-82ef-67b3c8f8a378 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.772628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9193cfd-132b-4639-82ef-67b3c8f8a378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.796988 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a9193cfd-132b-4639-82ef-67b3c8f8a378 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:39.798228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a9193cfd-132b-4639-82ef-67b3c8f8a378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:39.807923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96999163-d54b-4726-9cf6-0def14e70c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:39.808905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96999163-d54b-4726-9cf6-0def14e70c13 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.808905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96999163-d54b-4726-9cf6-0def14e70c13 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.809093 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96999163-d54b-4726-9cf6-0def14e70c13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.821186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96999163-d54b-4726-9cf6-0def14e70c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-96999163-d54b-4726-9cf6-0def14e70c13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:39.825999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:39.826940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:39.827759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 467/916] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Aug 8 16:41:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:39.833057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96999163-d54b-4726-9cf6-0def14e70c13 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:39.881835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 450/917] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:39.908057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9193cfd-132b-4639-82ef-67b3c8f8a378 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:39.942598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:39.950096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96999163-d54b-4726-9cf6-0def14e70c13 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:39.988796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2f05d-a7fa-4b67-b878-6ea7b76a96ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:39.989403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2f05d-a7fa-4b67-b878-6ea7b76a96ce None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:39.989679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2f05d-a7fa-4b67-b878-6ea7b76a96ce None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:39.990103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2f05d-a7fa-4b67-b878-6ea7b76a96ce None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:39.991271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2f05d-a7fa-4b67-b878-6ea7b76a96ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:39.991575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2f05d-a7fa-4b67-b878-6ea7b76a96ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:39.993064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2f05d-a7fa-4b67-b878-6ea7b76a96ce None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.001175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:40.003257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 468/918] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Aug 8 16:41:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:40.015041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2f05d-a7fa-4b67-b878-6ea7b76a96ce None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:40.018293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9193cfd-132b-4639-82ef-67b3c8f8a378 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:40.057389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adedf6-92af-4f0f-8728-c5731c041ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:40.058045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adedf6-92af-4f0f-8728-c5731c041ea6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:40.058320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adedf6-92af-4f0f-8728-c5731c041ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:40.058589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adedf6-92af-4f0f-8728-c5731c041ea6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:40.059163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adedf6-92af-4f0f-8728-c5731c041ea6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:40.060516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adedf6-92af-4f0f-8728-c5731c041ea6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-14adedf6-92af-4f0f-8728-c5731c041ea6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:40.069129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:40.071650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:40.073666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 451/919] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:41:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:40.077828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adedf6-92af-4f0f-8728-c5731c041ea6 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:40.124879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2f05d-a7fa-4b67-b878-6ea7b76a96ce None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:40.163031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027139d-005c-4556-bd51-f7f452dbd0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:40.163660 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027139d-005c-4556-bd51-f7f452dbd0c8 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:40.163936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027139d-005c-4556-bd51-f7f452dbd0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:40.164305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027139d-005c-4556-bd51-f7f452dbd0c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:40.176776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027139d-005c-4556-bd51-f7f452dbd0c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:40.178996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027139d-005c-4556-bd51-f7f452dbd0c8 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.180295 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adedf6-92af-4f0f-8728-c5731c041ea6 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-9027139d-005c-4556-bd51-f7f452dbd0c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:40.197687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:40.199617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:40.200739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:40.200739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:40.200739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:40.200739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 469/920] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:40.206648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027139d-005c-4556-bd51-f7f452dbd0c8 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:40.221596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccd35c-ac3f-44e3-ab6e-1ea297ff0d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:40.222388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd35c-ac3f-44e3-ab6e-1ea297ff0d75 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:40.222836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd35c-ac3f-44e3-ab6e-1ea297ff0d75 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:40.223235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd35c-ac3f-44e3-ab6e-1ea297ff0d75 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:40.223927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccd35c-ac3f-44e3-ab6e-1ea297ff0d75 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:40.224336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccd35c-ac3f-44e3-ab6e-1ea297ff0d75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:40.225043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:40.226482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccd35c-ac3f-44e3-ab6e-1ea297ff0d75 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.231718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 452/921] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:41:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:40.237343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccd35c-ac3f-44e3-ab6e-1ea297ff0d75 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:40.253860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:40.342591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccd35c-ac3f-44e3-ab6e-1ea297ff0d75 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:40.343389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027139d-005c-4556-bd51-f7f452dbd0c8 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:40.383504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc24a7-a4c4-4c34-98da-1837dc98a922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:40.384226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc24a7-a4c4-4c34-98da-1837dc98a922 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:40.384423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc24a7-a4c4-4c34-98da-1837dc98a922 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:40.384694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc24a7-a4c4-4c34-98da-1837dc98a922 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:40.384978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08239e2-62e9-486c-874c-2c339a4c16a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:40.385334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc24a7-a4c4-4c34-98da-1837dc98a922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:40.389823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc24a7-a4c4-4c34-98da-1837dc98a922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:40.389992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08239e2-62e9-486c-874c-2c339a4c16a6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:40.389992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08239e2-62e9-486c-874c-2c339a4c16a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:40.389992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08239e2-62e9-486c-874c-2c339a4c16a6 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:40.389992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08239e2-62e9-486c-874c-2c339a4c16a6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:40.390580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc24a7-a4c4-4c34-98da-1837dc98a922 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.390710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08239e2-62e9-486c-874c-2c339a4c16a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:40.390710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08239e2-62e9-486c-874c-2c339a4c16a6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.401368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 453/922] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Aug 8 16:41:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:40.403028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08239e2-62e9-486c-874c-2c339a4c16a6 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:40.405041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 470/923] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:40.410664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc24a7-a4c4-4c34-98da-1837dc98a922 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:40.488081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:40.509171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:40.533971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc24a7-a4c4-4c34-98da-1837dc98a922 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:40.535244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08239e2-62e9-486c-874c-2c339a4c16a6 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:40.573097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc7dd-4f1c-4910-bece-8fc65a6e18b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:40.573716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc7dd-4f1c-4910-bece-8fc65a6e18b0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:40.574020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc7dd-4f1c-4910-bece-8fc65a6e18b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:40.574286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc7dd-4f1c-4910-bece-8fc65a6e18b0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:40.578990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87000d71-bf24-4521-bd91-ccf4f9bb8853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:40.579592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87000d71-bf24-4521-bd91-ccf4f9bb8853 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:40.579862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87000d71-bf24-4521-bd91-ccf4f9bb8853 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:40.580128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87000d71-bf24-4521-bd91-ccf4f9bb8853 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:40.580907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87000d71-bf24-4521-bd91-ccf4f9bb8853 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:40.581738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bc7dd-4f1c-4910-bece-8fc65a6e18b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:40.582433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87000d71-bf24-4521-bd91-ccf4f9bb8853 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.589086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bc7dd-4f1c-4910-bece-8fc65a6e18b0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-955bc7dd-4f1c-4910-bece-8fc65a6e18b0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:40.604882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:40.605823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:40.607103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 471/924] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:41:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:40.611616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc7dd-4f1c-4910-bece-8fc65a6e18b0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:40.663875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 454/925] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:41:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:41:40.728522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc7dd-4f1c-4910-bece-8fc65a6e18b0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:40.758374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:40.773750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e9938-6ff6-4cc8-8867-e1c21a71ef89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:40.774394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e9938-6ff6-4cc8-8867-e1c21a71ef89 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:40.782032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e9938-6ff6-4cc8-8867-e1c21a71ef89 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:40.782032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e9938-6ff6-4cc8-8867-e1c21a71ef89 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:40.782032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e9938-6ff6-4cc8-8867-e1c21a71ef89 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:40.782032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e9938-6ff6-4cc8-8867-e1c21a71ef89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:40.782032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e9938-6ff6-4cc8-8867-e1c21a71ef89 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.799869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 472/926] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Aug 8 16:41:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:40.806916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87000d71-bf24-4521-bd91-ccf4f9bb8853 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:40.920762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87000d71-bf24-4521-bd91-ccf4f9bb8853 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:40.960235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a49fb-998a-4813-aa0b-4db06df1369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:40.961196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a49fb-998a-4813-aa0b-4db06df1369f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:40.961612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a49fb-998a-4813-aa0b-4db06df1369f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:40.962009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a49fb-998a-4813-aa0b-4db06df1369f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:40.964622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:40.975933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a49fb-998a-4813-aa0b-4db06df1369f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:40.977441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a49fb-998a-4813-aa0b-4db06df1369f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-239a49fb-998a-4813-aa0b-4db06df1369f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:40.991929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:40.992932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:40.993854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:40.993854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:40.993854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:40.993854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 455/927] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:41.000095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e9938-6ff6-4cc8-8867-e1c21a71ef89 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:41.108070 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e9938-6ff6-4cc8-8867-e1c21a71ef89 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:41.143293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bb1ff-7169-4a77-b484-504f48062da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:41.143886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bb1ff-7169-4a77-b484-504f48062da4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.144158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bb1ff-7169-4a77-b484-504f48062da4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.144423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bb1ff-7169-4a77-b484-504f48062da4 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.145059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bb1ff-7169-4a77-b484-504f48062da4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:41.145352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bb1ff-7169-4a77-b484-504f48062da4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:41.146846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bb1ff-7169-4a77-b484-504f48062da4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:41.158641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 473/928] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:41.164660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a49fb-998a-4813-aa0b-4db06df1369f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:41.269187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a49fb-998a-4813-aa0b-4db06df1369f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:41.308719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44504e3-db50-4483-8d0b-b67f5147bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:41.309836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44504e3-db50-4483-8d0b-b67f5147bc2a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.310324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44504e3-db50-4483-8d0b-b67f5147bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.310815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44504e3-db50-4483-8d0b-b67f5147bc2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.318066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44504e3-db50-4483-8d0b-b67f5147bc2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:41.319868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44504e3-db50-4483-8d0b-b67f5147bc2a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:41.326853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb3073-8915-4188-8617-fd5e72b3ec94 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.327136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb3073-8915-4188-8617-fd5e72b3ec94 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.327409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb3073-8915-4188-8617-fd5e72b3ec94 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.329099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 474/929] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-d44504e3-db50-4483-8d0b-b67f5147bc2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:41.332986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:41.333909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:41.334788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:41.334788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:41.334788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:41.334788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:41.334788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:41.334788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:41.334788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:41.334788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:41.335197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac23f-d7a1-497c-8514-a65a95ae30b0 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.335197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac23f-d7a1-497c-8514-a65a95ae30b0 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.335197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac23f-d7a1-497c-8514-a65a95ae30b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.337257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 456/930] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:41.343216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44504e3-db50-4483-8d0b-b67f5147bc2a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:41.354787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-faeac23f-d7a1-497c-8514-a65a95ae30b0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:41.356309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-faeac23f-d7a1-497c-8514-a65a95ae30b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:41.394234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:41.402134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:41.438054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 475/931] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:41.462530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeac23f-d7a1-497c-8514-a65a95ae30b0 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:41.472183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44504e3-db50-4483-8d0b-b67f5147bc2a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:41.519355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee46fe6-580b-451d-a3e6-7a0a80cfbd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:41.520801 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee46fe6-580b-451d-a3e6-7a0a80cfbd1c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.522196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee46fe6-580b-451d-a3e6-7a0a80cfbd1c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.522676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee46fe6-580b-451d-a3e6-7a0a80cfbd1c None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.524222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee46fe6-580b-451d-a3e6-7a0a80cfbd1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:41.524756 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee46fe6-580b-451d-a3e6-7a0a80cfbd1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:41.526906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee46fe6-580b-451d-a3e6-7a0a80cfbd1c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:41.544145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 457/932] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:41.550295 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee46fe6-580b-451d-a3e6-7a0a80cfbd1c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:41.574737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeac23f-d7a1-497c-8514-a65a95ae30b0 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:41.621559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdcd7f-6d06-49c4-9643-2a502e26553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:41.622217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdcd7f-6d06-49c4-9643-2a502e26553a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.622499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdcd7f-6d06-49c4-9643-2a502e26553a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.622858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdcd7f-6d06-49c4-9643-2a502e26553a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.623345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdcd7f-6d06-49c4-9643-2a502e26553a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:41.624965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdcd7f-6d06-49c4-9643-2a502e26553a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-32cdcd7f-6d06-49c4-9643-2a502e26553a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:41.635041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:41.636053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 16:41:41.636923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:41.645934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 476/933] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Aug 8 16:41:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:41.645934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdcd7f-6d06-49c4-9643-2a502e26553a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:41.688630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:41.695912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee46fe6-580b-451d-a3e6-7a0a80cfbd1c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:41.705976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:41.748339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a2ff1-10d3-4be8-b230-445565a843c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:41.749288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a2ff1-10d3-4be8-b230-445565a843c6 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.749566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a2ff1-10d3-4be8-b230-445565a843c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.749864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a2ff1-10d3-4be8-b230-445565a843c6 None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/19cf8bdae18e46ac8217c021765a2a78/roles/3455a742ed0a4787a4e60d7f104a586a` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.752038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a2ff1-10d3-4be8-b230-445565a843c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=19cf8bdae18e46ac8217c021765a2a78, role_id=3455a742ed0a4787a4e60d7f104a586a)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:41.753454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a2ff1-10d3-4be8-b230-445565a843c6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:41.763388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdcd7f-6d06-49c4-9643-2a502e26553a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:41.799689 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038818e-c6f1-43ee-bab1-31545fbcd941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:41.800505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038818e-c6f1-43ee-bab1-31545fbcd941 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.800951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038818e-c6f1-43ee-bab1-31545fbcd941 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.801367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038818e-c6f1-43ee-bab1-31545fbcd941 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.802160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038818e-c6f1-43ee-bab1-31545fbcd941 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:41.803112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038818e-c6f1-43ee-bab1-31545fbcd941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:41.804624 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038818e-c6f1-43ee-bab1-31545fbcd941 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:41.815783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 477/934] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:41.816818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038818e-c6f1-43ee-bab1-31545fbcd941 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:41.845820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 458/935] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:41:41 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/19cf8bdae18e46ac8217c021765a2a78/roles/3455a742ed0a4787a4e60d7f104a586a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:41:41.845820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2785a-5b8e-4178-a3a7-106ca4c6e358 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.845820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2785a-5b8e-4178-a3a7-106ca4c6e358 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.845820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2785a-5b8e-4178-a3a7-106ca4c6e358 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.845820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 459/936] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:41.848610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d67c1-2aa0-4799-b003-2ffcd89d64b5 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:41.848973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d67c1-2aa0-4799-b003-2ffcd89d64b5 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:41.849153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d67c1-2aa0-4799-b003-2ffcd89d64b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:41.873419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8a1d67c1-2aa0-4799-b003-2ffcd89d64b5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:41.874727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8a1d67c1-2aa0-4799-b003-2ffcd89d64b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:41.944898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:41.954113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038818e-c6f1-43ee-bab1-31545fbcd941 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:41.975290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 460/937] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:41.998226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d67c1-2aa0-4799-b003-2ffcd89d64b5 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:42.015651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c83c0-7578-4b91-885c-8e25ccddef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:42.016264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c83c0-7578-4b91-885c-8e25ccddef02 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:42.016537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c83c0-7578-4b91-885c-8e25ccddef02 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:42.016801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c83c0-7578-4b91-885c-8e25ccddef02 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:42.017431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c83c0-7578-4b91-885c-8e25ccddef02 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:42.017731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c83c0-7578-4b91-885c-8e25ccddef02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:42.019122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c83c0-7578-4b91-885c-8e25ccddef02 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:42.032938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 478/938] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:42.039266 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c83c0-7578-4b91-885c-8e25ccddef02 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:42.124864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:42.131787 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d67c1-2aa0-4799-b003-2ffcd89d64b5 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:42.147693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c83c0-7578-4b91-885c-8e25ccddef02 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:42.168497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b96f4-32f4-4da4-94ca-118afea6ec92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:42.169131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b96f4-32f4-4da4-94ca-118afea6ec92 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:42.169410 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b96f4-32f4-4da4-94ca-118afea6ec92 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:42.169681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b96f4-32f4-4da4-94ca-118afea6ec92 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:42.170344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b96f4-32f4-4da4-94ca-118afea6ec92 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:42.171814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b96f4-32f4-4da4-94ca-118afea6ec92 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-112b96f4-32f4-4da4-94ca-118afea6ec92 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:42.181418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:42.182436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:42.183901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 461/939] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:42.185130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e804fdc-86c5-4047-8dba-64f9b1ad07bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:42.185737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e804fdc-86c5-4047-8dba-64f9b1ad07bf None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:42.186041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e804fdc-86c5-4047-8dba-64f9b1ad07bf None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:42.186398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e804fdc-86c5-4047-8dba-64f9b1ad07bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:42.187154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e804fdc-86c5-4047-8dba-64f9b1ad07bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:42.188695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e804fdc-86c5-4047-8dba-64f9b1ad07bf None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:42.189003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b96f4-32f4-4da4-94ca-118afea6ec92 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:42.257924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 479/940] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:41:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:41:42.311110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b96f4-32f4-4da4-94ca-118afea6ec92 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:42.366817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47559e-e37d-4d6b-8f81-af5e1830d099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:42.367488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47559e-e37d-4d6b-8f81-af5e1830d099 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:42.367737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47559e-e37d-4d6b-8f81-af5e1830d099 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:42.367953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47559e-e37d-4d6b-8f81-af5e1830d099 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:42.368589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47559e-e37d-4d6b-8f81-af5e1830d099 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:42.369045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47559e-e37d-4d6b-8f81-af5e1830d099 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:42.370618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47559e-e37d-4d6b-8f81-af5e1830d099 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:42.376554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 462/941] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:42.382660 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e804fdc-86c5-4047-8dba-64f9b1ad07bf None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:42.407545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:42.495525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e804fdc-86c5-4047-8dba-64f9b1ad07bf None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:42.531967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879e8cb-a4e2-463e-b57c-9c71fccdf447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:42.532575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e8cb-a4e2-463e-b57c-9c71fccdf447 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:42.532853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e8cb-a4e2-463e-b57c-9c71fccdf447 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:42.533137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e8cb-a4e2-463e-b57c-9c71fccdf447 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:42.533754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879e8cb-a4e2-463e-b57c-9c71fccdf447 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:42.534083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879e8cb-a4e2-463e-b57c-9c71fccdf447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:42.535453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879e8cb-a4e2-463e-b57c-9c71fccdf447 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:42.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 480/942] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:42.546982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47559e-e37d-4d6b-8f81-af5e1830d099 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:42.571719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:42.666501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47559e-e37d-4d6b-8f81-af5e1830d099 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:42.701927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110a876-3584-4cd6-a964-f5c85125af25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:42.702537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110a876-3584-4cd6-a964-f5c85125af25 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:42.702812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110a876-3584-4cd6-a964-f5c85125af25 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:42.703080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110a876-3584-4cd6-a964-f5c85125af25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:42.703964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110a876-3584-4cd6-a964-f5c85125af25 None admin] RBAC: Adding query filter params (interface=public, service_id=30eff20e680f4fb3b3791d0f6f77a0ff, region_id=RegionOne) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:42.704274 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110a876-3584-4cd6-a964-f5c85125af25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:42.705636 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110a876-3584-4cd6-a964-f5c85125af25 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:42.712432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 463/943] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Aug 8 16:41:42 2022] GET /identity/v3/endpoints?service_id=30eff20e680f4fb3b3791d0f6f77a0ff&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:42.718901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879e8cb-a4e2-463e-b57c-9c71fccdf447 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:42.804177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:42.821697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879e8cb-a4e2-463e-b57c-9c71fccdf447 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:42.855896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c14937-079b-42a1-875b-bb99f160a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:42.856518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c14937-079b-42a1-875b-bb99f160a390 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:42.856821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c14937-079b-42a1-875b-bb99f160a390 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:42.857094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c14937-079b-42a1-875b-bb99f160a390 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:42.864499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c14937-079b-42a1-875b-bb99f160a390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:42.864499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c14937-079b-42a1-875b-bb99f160a390 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:42.865346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 481/944] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Aug 8 16:41:42 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:41:42.919989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff9eda-25bc-4d0a-a4b3-9a9b7a11b084 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:42.920172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff9eda-25bc-4d0a-a4b3-9a9b7a11b084 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:42.920441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff9eda-25bc-4d0a-a4b3-9a9b7a11b084 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:42.922209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 464/945] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:42.927933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89892d06-d1ce-492e-8817-9bfde9d02579 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:42.928207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89892d06-d1ce-492e-8817-9bfde9d02579 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:42.928479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89892d06-d1ce-492e-8817-9bfde9d02579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:42.948419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-89892d06-d1ce-492e-8817-9bfde9d02579 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:42.949628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-89892d06-d1ce-492e-8817-9bfde9d02579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:43.028409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 482/946] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:43.052187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff9eda-25bc-4d0a-a4b3-9a9b7a11b084 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:43.144843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:43.161919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff9eda-25bc-4d0a-a4b3-9a9b7a11b084 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:43.198456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:43.198990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:43.199267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:43.199533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:43.200173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:43.201592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:43.204596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING keystone.server.flask.common [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 16:41:43.259356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.sql.core [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 16:41:43.259356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 16:41:43.259356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [parameters: {'id': 'f3ddd95e42a549e88ab8e7dcba9f0401', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 16:41:43.259356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92261) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f3ddd95e42a549e88ab8e7dcba9f0401.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f3ddd95e42a549e88ab8e7dcba9f0401. Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:41:43.282263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application result.read() Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 16:41:43.283159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application yield resource Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise self.value Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application session.commit() Aug 08 16:41:43.284190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.session.flush() Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application compat.raise_( Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 16:41:43.285075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 16:41:43.286003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application result.read() Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:41:43.286840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [parameters: {'id': 'f3ddd95e42a549e88ab8e7dcba9f0401', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:43.287707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:43.288507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f3ddd95e42a549e88ab8e7dcba9f0401. Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:43.291140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 465/947] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Aug 8 16:41:43 2022] POST /identity/v3/projects => generated 262 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:43.292466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89892d06-d1ce-492e-8817-9bfde9d02579 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:43.384377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:43.399883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89892d06-d1ce-492e-8817-9bfde9d02579 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:43.437152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9759bd1-44c0-4833-8ee1-3caf3fc82271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:43.437814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9759bd1-44c0-4833-8ee1-3caf3fc82271 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:43.438069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9759bd1-44c0-4833-8ee1-3caf3fc82271 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:43.438337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9759bd1-44c0-4833-8ee1-3caf3fc82271 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:43.445618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9759bd1-44c0-4833-8ee1-3caf3fc82271 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:43.447099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9759bd1-44c0-4833-8ee1-3caf3fc82271 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-f9759bd1-44c0-4833-8ee1-3caf3fc82271 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:43.459036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:43.460011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:43.460987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:43.460987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:43.460987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:43.460987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:43.460987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:43.460987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:43.460987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:43.460987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:43.460987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 483/948] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:43.466867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:43.573216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e7aa1-59d5-42fc-9e7b-850fd9a6ef2c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:43.608596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c593a3-1596-4b9e-bf02-b606b81d25ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:43.609210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c593a3-1596-4b9e-bf02-b606b81d25ef None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:43.609481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c593a3-1596-4b9e-bf02-b606b81d25ef None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:43.609746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c593a3-1596-4b9e-bf02-b606b81d25ef None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:43.610454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c593a3-1596-4b9e-bf02-b606b81d25ef None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:43.610957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c593a3-1596-4b9e-bf02-b606b81d25ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:43.612597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c593a3-1596-4b9e-bf02-b606b81d25ef None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:43.627033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:43.636925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 466/949] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:43.980029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a1d56-9641-494e-bcba-3b1a8ee4eca6 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:43.980431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a1d56-9641-494e-bcba-3b1a8ee4eca6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:43.980626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a1d56-9641-494e-bcba-3b1a8ee4eca6 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:43.982203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 484/950] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:43.987523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202a04-464f-4c95-895c-db360e6b433f None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:43.987807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202a04-464f-4c95-895c-db360e6b433f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:43.988082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202a04-464f-4c95-895c-db360e6b433f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.008518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-79202a04-464f-4c95-895c-db360e6b433f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:44.010066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-79202a04-464f-4c95-895c-db360e6b433f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:44.084387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 467/951] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:44.107406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a1d56-9641-494e-bcba-3b1a8ee4eca6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:44.214348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a1d56-9641-494e-bcba-3b1a8ee4eca6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:44.249688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9706a36-072e-4692-91c2-1b03c313f2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:44.250272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9706a36-072e-4692-91c2-1b03c313f2d5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:44.250548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9706a36-072e-4692-91c2-1b03c313f2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:44.250814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9706a36-072e-4692-91c2-1b03c313f2d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.260986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9706a36-072e-4692-91c2-1b03c313f2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-f9706a36-072e-4692-91c2-1b03c313f2d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:44.265719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:44.268025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 485/952] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Aug 8 16:41:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:44.272747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:44.279141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202a04-464f-4c95-895c-db360e6b433f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:44.389207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202a04-464f-4c95-895c-db360e6b433f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:44.419789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:44.425779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f173d-5fe5-444b-a913-a87897d6d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:44.426446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f173d-5fe5-444b-a913-a87897d6d5da None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:44.426724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f173d-5fe5-444b-a913-a87897d6d5da None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:44.426993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f173d-5fe5-444b-a913-a87897d6d5da None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.427736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f173d-5fe5-444b-a913-a87897d6d5da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:44.428034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f173d-5fe5-444b-a913-a87897d6d5da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:44.429405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f173d-5fe5-444b-a913-a87897d6d5da None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:44.437958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 468/953] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Aug 8 16:41:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:44.444033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9706a36-072e-4692-91c2-1b03c313f2d5 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:44.486182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f1fef-093d-4e1d-9a9c-68b41954d8dc None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:44.486697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f1fef-093d-4e1d-9a9c-68b41954d8dc None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:44.486803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f1fef-093d-4e1d-9a9c-68b41954d8dc None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.488360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 469/954] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:41:44.494315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3f5a1-d02e-4b86-8b7a-ce16f01bf85d None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:44.494850 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3f5a1-d02e-4b86-8b7a-ce16f01bf85d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:44.495335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3f5a1-d02e-4b86-8b7a-ce16f01bf85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.516440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a5c3f5a1-d02e-4b86-8b7a-ce16f01bf85d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:44.518113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a5c3f5a1-d02e-4b86-8b7a-ce16f01bf85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:44.555852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9706a36-072e-4692-91c2-1b03c313f2d5 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:44.597880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ce3fa-5512-4d2b-b918-a7713e528b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:44.598514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ce3fa-5512-4d2b-b918-a7713e528b1c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:44.598789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ce3fa-5512-4d2b-b918-a7713e528b1c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:44.599687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ce3fa-5512-4d2b-b918-a7713e528b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.605921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 470/955] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:44.614291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ce3fa-5512-4d2b-b918-a7713e528b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:44.615863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ce3fa-5512-4d2b-b918-a7713e528b1c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:44.627976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3f5a1-d02e-4b86-8b7a-ce16f01bf85d None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-681ce3fa-5512-4d2b-b918-a7713e528b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:44.632405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:44.634656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:44.636603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:44.636603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:44.636603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:44.636603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 486/956] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:44.642940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ce3fa-5512-4d2b-b918-a7713e528b1c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:44.694712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:44.746716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3f5a1-d02e-4b86-8b7a-ce16f01bf85d None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:44.747616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ce3fa-5512-4d2b-b918-a7713e528b1c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:44.772302 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:44.782732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca960bc-1685-4c61-ae3a-11ed957c1c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:44.783504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca960bc-1685-4c61-ae3a-11ed957c1c52 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:44.783926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca960bc-1685-4c61-ae3a-11ed957c1c52 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:44.784264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca960bc-1685-4c61-ae3a-11ed957c1c52 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.784975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca960bc-1685-4c61-ae3a-11ed957c1c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:44.785169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a481f-f1b2-439f-8a79-4b39d35c564f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:44.785386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca960bc-1685-4c61-ae3a-11ed957c1c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:44.785725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a481f-f1b2-439f-8a79-4b39d35c564f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:44.786040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a481f-f1b2-439f-8a79-4b39d35c564f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:44.786313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a481f-f1b2-439f-8a79-4b39d35c564f None admin] PATH_INFO: `/v3/services/compute` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.786954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a481f-f1b2-439f-8a79-4b39d35c564f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:44.787247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca960bc-1685-4c61-ae3a-11ed957c1c52 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:44.788298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a481f-f1b2-439f-8a79-4b39d35c564f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-fc0a481f-f1b2-439f-8a79-4b39d35c564f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:44.798653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:44.800462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:44.801574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 471/957] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:44.809811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 487/958] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:44.809973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a481f-f1b2-439f-8a79-4b39d35c564f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:44.811216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca960bc-1685-4c61-ae3a-11ed957c1c52 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:44.915456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca960bc-1685-4c61-ae3a-11ed957c1c52 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:44.919012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a481f-f1b2-439f-8a79-4b39d35c564f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:44.928565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:44.953052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea004e0-fe9d-46f7-af03-fa69f582dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:44.953844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea004e0-fe9d-46f7-af03-fa69f582dd8d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:44.954235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea004e0-fe9d-46f7-af03-fa69f582dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:44.954597 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea004e0-fe9d-46f7-af03-fa69f582dd8d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.957269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bab0c5-89ab-479e-bdbe-e94154029296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:44.957901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bab0c5-89ab-479e-bdbe-e94154029296 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:44.958182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bab0c5-89ab-479e-bdbe-e94154029296 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:44.958478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bab0c5-89ab-479e-bdbe-e94154029296 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:44.959112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bab0c5-89ab-479e-bdbe-e94154029296 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:44.959413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bab0c5-89ab-479e-bdbe-e94154029296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:44.960766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bab0c5-89ab-479e-bdbe-e94154029296 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:44.962170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea004e0-fe9d-46f7-af03-fa69f582dd8d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:44.963594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea004e0-fe9d-46f7-af03-fa69f582dd8d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:44.967227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 472/959] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:44.973995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bab0c5-89ab-479e-bdbe-e94154029296 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-6ea004e0-fe9d-46f7-af03-fa69f582dd8d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:44.982219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:44.984005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:44.985201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 488/960] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:41:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:44.993814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea004e0-fe9d-46f7-af03-fa69f582dd8d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:45.029299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:45.083616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bab0c5-89ab-479e-bdbe-e94154029296 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:45.092657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea004e0-fe9d-46f7-af03-fa69f582dd8d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:45.117912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8827f4f-cabe-41d8-9277-e2c01cc848a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:45.118570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8827f4f-cabe-41d8-9277-e2c01cc848a7 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.118848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8827f4f-cabe-41d8-9277-e2c01cc848a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.119121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8827f4f-cabe-41d8-9277-e2c01cc848a7 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.119746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8827f4f-cabe-41d8-9277-e2c01cc848a7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:45.120046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8827f4f-cabe-41d8-9277-e2c01cc848a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:45.121396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8827f4f-cabe-41d8-9277-e2c01cc848a7 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:45.133818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 473/961] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:45.133818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0c1bc-989e-4a24-8e61-b38aa13f344d None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.133818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0c1bc-989e-4a24-8e61-b38aa13f344d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.133818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0c1bc-989e-4a24-8e61-b38aa13f344d None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.133818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 474/962] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:45.136082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6bca-8237-48be-9a8a-6ee1294018c4 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.136509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6bca-8237-48be-9a8a-6ee1294018c4 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.136796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6bca-8237-48be-9a8a-6ee1294018c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.140344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef5ef3-4409-49ec-80ff-e79b83f48e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:45.141314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef5ef3-4409-49ec-80ff-e79b83f48e11 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.141834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef5ef3-4409-49ec-80ff-e79b83f48e11 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.142339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef5ef3-4409-49ec-80ff-e79b83f48e11 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.143307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef5ef3-4409-49ec-80ff-e79b83f48e11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:45.143839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef5ef3-4409-49ec-80ff-e79b83f48e11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:45.145536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef5ef3-4409-49ec-80ff-e79b83f48e11 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:45.163125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-cbfb6bca-8237-48be-9a8a-6ee1294018c4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:45.165224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cbfb6bca-8237-48be-9a8a-6ee1294018c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:45.168908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 489/963] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Aug 8 16:41:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:45.181869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef5ef3-4409-49ec-80ff-e79b83f48e11 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:45.209735 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:45.269049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 475/964] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:45.291830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb6bca-8237-48be-9a8a-6ee1294018c4 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:45.295753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef5ef3-4409-49ec-80ff-e79b83f48e11 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:45.345148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c179bd-4103-49ed-a391-9f12f2997490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:45.346108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c179bd-4103-49ed-a391-9f12f2997490 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.346394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c179bd-4103-49ed-a391-9f12f2997490 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.346667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c179bd-4103-49ed-a391-9f12f2997490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.350183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:45.410042 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb6bca-8237-48be-9a8a-6ee1294018c4 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:45.431390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c179bd-4103-49ed-a391-9f12f2997490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:45.433506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c179bd-4103-49ed-a391-9f12f2997490 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:45.462018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990610a1-c01f-4fe4-bde3-79e7b106f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:45.462664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990610a1-c01f-4fe4-bde3-79e7b106f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.462940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990610a1-c01f-4fe4-bde3-79e7b106f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.463214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990610a1-c01f-4fe4-bde3-79e7b106f6f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.464052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990610a1-c01f-4fe4-bde3-79e7b106f6f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:45.465444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:45.466799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990610a1-c01f-4fe4-bde3-79e7b106f6f6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-990610a1-c01f-4fe4-bde3-79e7b106f6f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:45.479879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:45.480922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:45.481749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:45.484577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 476/965] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:45.490941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990610a1-c01f-4fe4-bde3-79e7b106f6f6 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:45.563716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 490/966] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:41:45.569457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c179bd-4103-49ed-a391-9f12f2997490 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:45.605188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990610a1-c01f-4fe4-bde3-79e7b106f6f6 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:45.615677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:45.643406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0d6c0-6611-4170-bee2-6de6b95fe507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:45.644176 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0d6c0-6611-4170-bee2-6de6b95fe507 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.644585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0d6c0-6611-4170-bee2-6de6b95fe507 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.645020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0d6c0-6611-4170-bee2-6de6b95fe507 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.646540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0d6c0-6611-4170-bee2-6de6b95fe507 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:45.647057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0d6c0-6611-4170-bee2-6de6b95fe507 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:45.648773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0d6c0-6611-4170-bee2-6de6b95fe507 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:45.654591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 477/967] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:45.667326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0d6c0-6611-4170-bee2-6de6b95fe507 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:45.683200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c179bd-4103-49ed-a391-9f12f2997490 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:45.715141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:45.723696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78ae69-61b6-4d9c-8bd4-4426ff5957bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:45.724718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78ae69-61b6-4d9c-8bd4-4426ff5957bc None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.725087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78ae69-61b6-4d9c-8bd4-4426ff5957bc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.729896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78ae69-61b6-4d9c-8bd4-4426ff5957bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.750010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78ae69-61b6-4d9c-8bd4-4426ff5957bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:45.752719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78ae69-61b6-4d9c-8bd4-4426ff5957bc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-ea78ae69-61b6-4d9c-8bd4-4426ff5957bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:45.774129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:45.775127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:45.776001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:45.776001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:45.776001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:45.782988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 491/968] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:45.785288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78ae69-61b6-4d9c-8bd4-4426ff5957bc None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:45.794149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0d6c0-6611-4170-bee2-6de6b95fe507 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:45.831360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8d928-63dc-4c6c-b843-aa92b27273ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:45.832057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8d928-63dc-4c6c-b843-aa92b27273ef None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.832338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8d928-63dc-4c6c-b843-aa92b27273ef None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.832777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8d928-63dc-4c6c-b843-aa92b27273ef None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.833226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8d928-63dc-4c6c-b843-aa92b27273ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:45.833655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8d928-63dc-4c6c-b843-aa92b27273ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:45.835114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8d928-63dc-4c6c-b843-aa92b27273ef None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:45.841059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 478/969] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:45.842437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98345d-115a-437d-9cfd-0dd775133445 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.842723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98345d-115a-437d-9cfd-0dd775133445 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.842995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98345d-115a-437d-9cfd-0dd775133445 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.844265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 479/970] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:45.849832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98345d-115a-437d-9cfd-0dd775133445 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:45.896458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:45.896647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78ae69-61b6-4d9c-8bd4-4426ff5957bc None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:45.930432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc9053-bc02-4b5b-a492-b61426e841d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:45.931029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc9053-bc02-4b5b-a492-b61426e841d6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.931303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc9053-bc02-4b5b-a492-b61426e841d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.931569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc9053-bc02-4b5b-a492-b61426e841d6 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.932199 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc9053-bc02-4b5b-a492-b61426e841d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:45.932494 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc9053-bc02-4b5b-a492-b61426e841d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:45.933893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc9053-bc02-4b5b-a492-b61426e841d6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:45.945929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 492/971] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:45.947254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4360468-0753-4ced-8422-57604d36f3ca None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:45.947529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4360468-0753-4ced-8422-57604d36f3ca None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:45.947797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4360468-0753-4ced-8422-57604d36f3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:45.965047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d4360468-0753-4ced-8422-57604d36f3ca None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:45.966238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d4360468-0753-4ced-8422-57604d36f3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:45.973232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98345d-115a-437d-9cfd-0dd775133445 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:45.999612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:46.010180 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67506d47-0a57-4c81-90b9-fd7501042dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:46.011653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67506d47-0a57-4c81-90b9-fd7501042dcf None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:46.011653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67506d47-0a57-4c81-90b9-fd7501042dcf None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:46.011653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67506d47-0a57-4c81-90b9-fd7501042dcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:46.011913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67506d47-0a57-4c81-90b9-fd7501042dcf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:46.013244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67506d47-0a57-4c81-90b9-fd7501042dcf None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:46.033103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 493/972] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:46.034053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4360468-0753-4ced-8422-57604d36f3ca None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:46.121909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 480/973] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:41:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:46.121909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67506d47-0a57-4c81-90b9-fd7501042dcf None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:46.155413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4360468-0753-4ced-8422-57604d36f3ca None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:46.214629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5508b4-7dab-4913-b43c-f02fbac3151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:46.215665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5508b4-7dab-4913-b43c-f02fbac3151a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:46.216228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5508b4-7dab-4913-b43c-f02fbac3151a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:46.216690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5508b4-7dab-4913-b43c-f02fbac3151a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:46.227134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5508b4-7dab-4913-b43c-f02fbac3151a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:46.229058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5508b4-7dab-4913-b43c-f02fbac3151a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:46.231353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:46.241763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67506d47-0a57-4c81-90b9-fd7501042dcf None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-ca5508b4-7dab-4913-b43c-f02fbac3151a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:46.246842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:46.254884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:46.256020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:46.256020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:46.256020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:46.256020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:46.256020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:46.256020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:46.256020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:46.256020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:46.257144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 494/974] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:46.258286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5508b4-7dab-4913-b43c-f02fbac3151a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:46.300609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:46.301191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:46.302613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:46.302738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:46.303305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:46.304248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:46.305420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:46.377030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.sql.core [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 16:41:46.377030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 16:41:46.377030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [parameters: {'user_id': 'd2f5218953ce4f8da57ced238e059dd3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 16:41:46.377030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92261) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application result.read() Aug 08 16:41:46.387773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:41:46.388705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application yield resource Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application next(self.gen) Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise self.value Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application session.commit() Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 16:41:46.390353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.session.flush() Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application compat.raise_( Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 16:41:46.391222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 16:41:46.392098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application conn.query(q) Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application result.read() Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 16:41:46.393000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd2f5218953ce4f8da57ced238e059dd3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:46.395307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 16:41:46.398066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 16:41:46.400442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 16:41:46.400442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 16:41:46.400442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 16:41:46.400442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 16:41:46.400442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 16:41:46.400442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:46.400442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 481/975] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Aug 8 16:41:46 2022] POST /identity/v3/users => generated 160 bytes in 275 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:46.401356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5508b4-7dab-4913-b43c-f02fbac3151a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:46.401503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:46.436722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ae209-28f8-43b6-989e-b06dad8a07d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:46.437307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ae209-28f8-43b6-989e-b06dad8a07d4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:46.437565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ae209-28f8-43b6-989e-b06dad8a07d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:46.437910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ae209-28f8-43b6-989e-b06dad8a07d4 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:46.438595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ae209-28f8-43b6-989e-b06dad8a07d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:46.438945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ae209-28f8-43b6-989e-b06dad8a07d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:46.440462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ae209-28f8-43b6-989e-b06dad8a07d4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:46.453818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 495/976] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:46.462012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ae209-28f8-43b6-989e-b06dad8a07d4 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:46.534239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa166ab-5f03-496a-8425-d162f5d7805e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:46.554142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:46.575339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8ede-d7e3-47f0-8ec2-58772f0c4a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:46.575902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ae209-28f8-43b6-989e-b06dad8a07d4 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:46.576209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8ede-d7e3-47f0-8ec2-58772f0c4a65 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:46.576583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8ede-d7e3-47f0-8ec2-58772f0c4a65 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:46.577023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8ede-d7e3-47f0-8ec2-58772f0c4a65 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:46.586524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd8ede-d7e3-47f0-8ec2-58772f0c4a65 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:46.594000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd8ede-d7e3-47f0-8ec2-58772f0c4a65 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-dddd8ede-d7e3-47f0-8ec2-58772f0c4a65 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:46.605057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:46.606080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 16:41:46.607020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:46.609603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 482/977] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:46.616432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8ede-d7e3-47f0-8ec2-58772f0c4a65 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:46.635253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8b6dd-f0c5-4456-b5c4-85d904ba30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:46.635828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8b6dd-f0c5-4456-b5c4-85d904ba30d0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:46.636104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8b6dd-f0c5-4456-b5c4-85d904ba30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:46.636371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8b6dd-f0c5-4456-b5c4-85d904ba30d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:46.637251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8b6dd-f0c5-4456-b5c4-85d904ba30d0 None admin] RBAC: Adding query filter params (role.id=3455a742ed0a4787a4e60d7f104a586a, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=19cf8bdae18e46ac8217c021765a2a78) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:46.637562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8b6dd-f0c5-4456-b5c4-85d904ba30d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:46.639118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8b6dd-f0c5-4456-b5c4-85d904ba30d0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:46.640450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:46.654330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 496/978] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:46 2022] GET /identity/v3/role_assignments?user.id=19cf8bdae18e46ac8217c021765a2a78&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=3455a742ed0a4787a4e60d7f104a586a => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:46.737568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8ede-d7e3-47f0-8ec2-58772f0c4a65 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:46.775784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101582ef-ce64-4062-a081-667f72eb59c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:46.776807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101582ef-ce64-4062-a081-667f72eb59c0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:46.777326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101582ef-ce64-4062-a081-667f72eb59c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:46.777829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101582ef-ce64-4062-a081-667f72eb59c0 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:46.778765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101582ef-ce64-4062-a081-667f72eb59c0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:46.779282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101582ef-ce64-4062-a081-667f72eb59c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:46.780957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101582ef-ce64-4062-a081-667f72eb59c0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:46.824919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 483/979] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Aug 8 16:41:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:47.209761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf3a55-48ec-492b-801c-22af904b0da9 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:47.210153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf3a55-48ec-492b-801c-22af904b0da9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:47.210347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf3a55-48ec-492b-801c-22af904b0da9 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:47.211808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 497/980] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:47.216943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301240d-5579-49c3-a553-6ed636724aac None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:47.217232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301240d-5579-49c3-a553-6ed636724aac None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:47.217503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301240d-5579-49c3-a553-6ed636724aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:47.236222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a301240d-5579-49c3-a553-6ed636724aac None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:47.237729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a301240d-5579-49c3-a553-6ed636724aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:47.312992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 484/981] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:47.339890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf3a55-48ec-492b-801c-22af904b0da9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:47.444958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf3a55-48ec-492b-801c-22af904b0da9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:47.478874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e5571-e5fb-4f86-9c65-ac97c5cb6cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:47.479476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e5571-e5fb-4f86-9c65-ac97c5cb6cad None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:47.479753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e5571-e5fb-4f86-9c65-ac97c5cb6cad None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:47.480024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e5571-e5fb-4f86-9c65-ac97c5cb6cad None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:47.480657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e5571-e5fb-4f86-9c65-ac97c5cb6cad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:47.482039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e5571-e5fb-4f86-9c65-ac97c5cb6cad None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:47.581820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 498/982] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:41:47 2022] POST /identity/v3/services => generated 242 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:48.178449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c6c4c-4f60-4f1a-9a3b-391a660e4b0c None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:48.178971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c6c4c-4f60-4f1a-9a3b-391a660e4b0c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:48.178971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c6c4c-4f60-4f1a-9a3b-391a660e4b0c None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:48.180451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 485/983] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:48.185408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2451573-8701-4218-b315-666259b801f4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:48.185767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2451573-8701-4218-b315-666259b801f4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:48.186141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2451573-8701-4218-b315-666259b801f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:48.204393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a2451573-8701-4218-b315-666259b801f4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:48.205575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a2451573-8701-4218-b315-666259b801f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:48.209861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:48.281507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 499/984] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:48.304231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c6c4c-4f60-4f1a-9a3b-391a660e4b0c None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:48.307210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:48.410266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c6c4c-4f60-4f1a-9a3b-391a660e4b0c None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:48.443975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e8ceb-4a5d-472a-a625-a8dd8bc204cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:48.444531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e8ceb-4a5d-472a-a625-a8dd8bc204cc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:48.444809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e8ceb-4a5d-472a-a625-a8dd8bc204cc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:48.445331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e8ceb-4a5d-472a-a625-a8dd8bc204cc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:48.445950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e8ceb-4a5d-472a-a625-a8dd8bc204cc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:48.447303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e8ceb-4a5d-472a-a625-a8dd8bc204cc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-f18e8ceb-4a5d-472a-a625-a8dd8bc204cc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:48.455931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:48.456953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:48.457825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:48.459864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 486/985] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:48.463521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2451573-8701-4218-b315-666259b801f4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:48.561210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2451573-8701-4218-b315-666259b801f4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:48.605719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e9714-c5ff-4b49-9717-64f2c2b74850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:48.606430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e9714-c5ff-4b49-9717-64f2c2b74850 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:48.606650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e9714-c5ff-4b49-9717-64f2c2b74850 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:48.606921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e9714-c5ff-4b49-9717-64f2c2b74850 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:48.607551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e9714-c5ff-4b49-9717-64f2c2b74850 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:48.607852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e9714-c5ff-4b49-9717-64f2c2b74850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:48.609207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e9714-c5ff-4b49-9717-64f2c2b74850 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:48.611027 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:48.614954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 500/986] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:48.620456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e8ceb-4a5d-472a-a625-a8dd8bc204cc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:48.688894 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:48.720093 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e0a58-45c2-466a-8de6-f8d056c71ca6 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:48.720473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e0a58-45c2-466a-8de6-f8d056c71ca6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:48.720645 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e0a58-45c2-466a-8de6-f8d056c71ca6 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:48.722020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 501/987] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:48.727807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd49de9-487d-4c99-95cf-8f66de266bfd None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:48.728133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd49de9-487d-4c99-95cf-8f66de266bfd None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:48.728348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd49de9-487d-4c99-95cf-8f66de266bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:48.728601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e8ceb-4a5d-472a-a625-a8dd8bc204cc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:48.747658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-bbd49de9-487d-4c99-95cf-8f66de266bfd None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:48.748820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bbd49de9-487d-4c99-95cf-8f66de266bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:48.763923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8397d-30d4-442e-afcb-dc4cefbaa4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:48.764532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8397d-30d4-442e-afcb-dc4cefbaa4bf None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:48.764811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8397d-30d4-442e-afcb-dc4cefbaa4bf None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:48.765080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8397d-30d4-442e-afcb-dc4cefbaa4bf None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:48.765708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8397d-30d4-442e-afcb-dc4cefbaa4bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:48.766048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8397d-30d4-442e-afcb-dc4cefbaa4bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:48.767407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8397d-30d4-442e-afcb-dc4cefbaa4bf None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:48.773018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 487/988] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:48.778663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8397d-30d4-442e-afcb-dc4cefbaa4bf None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:48.819742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 502/989] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:48.841893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd49de9-487d-4c99-95cf-8f66de266bfd None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:48.879899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8397d-30d4-442e-afcb-dc4cefbaa4bf None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:48.910726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:48.916846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ce63d-92b2-4b7d-a34e-749e11fc238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:48.917455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ce63d-92b2-4b7d-a34e-749e11fc238a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:48.917748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ce63d-92b2-4b7d-a34e-749e11fc238a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:48.918032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ce63d-92b2-4b7d-a34e-749e11fc238a None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:48.918811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ce63d-92b2-4b7d-a34e-749e11fc238a None admin] RBAC: Adding query filter params (interface=admin, service_id=30eff20e680f4fb3b3791d0f6f77a0ff, region_id=RegionOne) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:48.919119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ce63d-92b2-4b7d-a34e-749e11fc238a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:48.920468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ce63d-92b2-4b7d-a34e-749e11fc238a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:48.929053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 488/990] 10.222.0.57 () {56 vars in 1249 bytes} [Mon Aug 8 16:41:48 2022] GET /identity/v3/endpoints?service_id=30eff20e680f4fb3b3791d0f6f77a0ff&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:48.931537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ce63d-92b2-4b7d-a34e-749e11fc238a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:48.944075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd49de9-487d-4c99-95cf-8f66de266bfd None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:48.973191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:48.979945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e590c-cf78-4295-9e3e-914dde87c062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:48.980569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e590c-cf78-4295-9e3e-914dde87c062 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:48.980833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e590c-cf78-4295-9e3e-914dde87c062 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:48.981100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e590c-cf78-4295-9e3e-914dde87c062 None admin] PATH_INFO: `/v3/services/image` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:48.981682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e590c-cf78-4295-9e3e-914dde87c062 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:48.983061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e590c-cf78-4295-9e3e-914dde87c062 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-1b3e590c-cf78-4295-9e3e-914dde87c062 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:48.993039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:48.994108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:48.994959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 503/991] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Aug 8 16:41:48 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:41:49.002286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e590c-cf78-4295-9e3e-914dde87c062 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:49.038012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ce63d-92b2-4b7d-a34e-749e11fc238a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:49.077613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed882b79-a4d3-4a77-af36-bdd30f1398a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:49.078253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed882b79-a4d3-4a77-af36-bdd30f1398a0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:49.078529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed882b79-a4d3-4a77-af36-bdd30f1398a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:49.078796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed882b79-a4d3-4a77-af36-bdd30f1398a0 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:49.079405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed882b79-a4d3-4a77-af36-bdd30f1398a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:49.080766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed882b79-a4d3-4a77-af36-bdd30f1398a0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:49.089829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 489/992] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Aug 8 16:41:48 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 16:41:49.107625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e590c-cf78-4295-9e3e-914dde87c062 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:49.144787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25524c49-32d4-4eff-8dd1-3ccefa031d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:49.145407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25524c49-32d4-4eff-8dd1-3ccefa031d86 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:49.146213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25524c49-32d4-4eff-8dd1-3ccefa031d86 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:49.146479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25524c49-32d4-4eff-8dd1-3ccefa031d86 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:49.147194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25524c49-32d4-4eff-8dd1-3ccefa031d86 None admin] RBAC: Adding query filter params (name=image) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:49.147497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25524c49-32d4-4eff-8dd1-3ccefa031d86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:49.148871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25524c49-32d4-4eff-8dd1-3ccefa031d86 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:49.159357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 504/993] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:49.161007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed882b79-a4d3-4a77-af36-bdd30f1398a0 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:49.226707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:49.268766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed882b79-a4d3-4a77-af36-bdd30f1398a0 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:49.304084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cef7c7-8ed9-4ef9-8a9f-c5a5b50b0855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:49.304692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cef7c7-8ed9-4ef9-8a9f-c5a5b50b0855 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:49.304970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cef7c7-8ed9-4ef9-8a9f-c5a5b50b0855 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:49.306711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cef7c7-8ed9-4ef9-8a9f-c5a5b50b0855 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:49.306711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cef7c7-8ed9-4ef9-8a9f-c5a5b50b0855 None admin] RBAC: Adding query filter params (type=image) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:49.306711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cef7c7-8ed9-4ef9-8a9f-c5a5b50b0855 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:49.307761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cef7c7-8ed9-4ef9-8a9f-c5a5b50b0855 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:49.312432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 490/994] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:49.638797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324d63e-7a91-4972-ad6f-71e1cd1dbd61 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:49.639345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324d63e-7a91-4972-ad6f-71e1cd1dbd61 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:49.639345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324d63e-7a91-4972-ad6f-71e1cd1dbd61 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:49.640921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 505/995] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:49.646341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfd5eb-4c5e-4720-8490-dbb3424dd40d None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:49.646628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfd5eb-4c5e-4720-8490-dbb3424dd40d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:49.646900 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfd5eb-4c5e-4720-8490-dbb3424dd40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:49.665172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-31dfd5eb-4c5e-4720-8490-dbb3424dd40d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:49.666376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-31dfd5eb-4c5e-4720-8490-dbb3424dd40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:49.735613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 491/996] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:49.757578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324d63e-7a91-4972-ad6f-71e1cd1dbd61 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:49.821111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:49.863083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324d63e-7a91-4972-ad6f-71e1cd1dbd61 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:49.897872 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b38272-9547-407c-977e-b263e840c410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:49.898492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b38272-9547-407c-977e-b263e840c410 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:49.898766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b38272-9547-407c-977e-b263e840c410 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:49.899034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b38272-9547-407c-977e-b263e840c410 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:49.899607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b38272-9547-407c-977e-b263e840c410 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:49.900952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b38272-9547-407c-977e-b263e840c410 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-99b38272-9547-407c-977e-b263e840c410 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:49.909465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:49.910670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:41:49.912147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:49.914117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 506/997] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:49.917113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfd5eb-4c5e-4720-8490-dbb3424dd40d None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:49.971241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:50.018125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfd5eb-4c5e-4720-8490-dbb3424dd40d None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:50.058643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20f961-7da9-4b8a-a125-13bc3c379e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:50.059257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20f961-7da9-4b8a-a125-13bc3c379e12 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:50.059536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20f961-7da9-4b8a-a125-13bc3c379e12 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:50.059804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20f961-7da9-4b8a-a125-13bc3c379e12 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:50.060436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20f961-7da9-4b8a-a125-13bc3c379e12 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:50.060737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20f961-7da9-4b8a-a125-13bc3c379e12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:50.062185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20f961-7da9-4b8a-a125-13bc3c379e12 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:50.072022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 492/998] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:50.073524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b38272-9547-407c-977e-b263e840c410 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:50.173676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b38272-9547-407c-977e-b263e840c410 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:50.201632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:50.213749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2481ab-d5de-46e1-ae16-6129e4011b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:50.214384 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2481ab-d5de-46e1-ae16-6129e4011b00 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:50.214661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2481ab-d5de-46e1-ae16-6129e4011b00 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:50.214929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2481ab-d5de-46e1-ae16-6129e4011b00 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:50.221722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2481ab-d5de-46e1-ae16-6129e4011b00 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:50.221722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2481ab-d5de-46e1-ae16-6129e4011b00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:50.221722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2481ab-d5de-46e1-ae16-6129e4011b00 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:50.222945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 507/999] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:50.229149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20f961-7da9-4b8a-a125-13bc3c379e12 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:50.330172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20f961-7da9-4b8a-a125-13bc3c379e12 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:50.360508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:50.378174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec976c-e561-439f-9328-a7eeacc117d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:50.378781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec976c-e561-439f-9328-a7eeacc117d4 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:50.379060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec976c-e561-439f-9328-a7eeacc117d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:50.379338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec976c-e561-439f-9328-a7eeacc117d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:50.380897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec976c-e561-439f-9328-a7eeacc117d4 None admin] RBAC: Adding query filter params (interface=public, service_id=ee273a6752df4637b2119f4cad385859, region_id=RegionOne) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:50.381205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec976c-e561-439f-9328-a7eeacc117d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:50.382594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec976c-e561-439f-9328-a7eeacc117d4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:50.389940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 493/1000] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Aug 8 16:41:50 2022] GET /identity/v3/endpoints?service_id=ee273a6752df4637b2119f4cad385859&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:50.393632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2481ab-d5de-46e1-ae16-6129e4011b00 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:50.495515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2481ab-d5de-46e1-ae16-6129e4011b00 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:50.528047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62542310-cbab-4ff4-ad12-767e9da63d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:50.528648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62542310-cbab-4ff4-ad12-767e9da63d6e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:50.528925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62542310-cbab-4ff4-ad12-767e9da63d6e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:50.529193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62542310-cbab-4ff4-ad12-767e9da63d6e None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:50.529805 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62542310-cbab-4ff4-ad12-767e9da63d6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:50.531158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62542310-cbab-4ff4-ad12-767e9da63d6e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:50.541908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 508/1001] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Aug 8 16:41:50 2022] GET /identity/v3/services => generated 1727 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:41:51.135590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b899d-f620-49fe-943b-aea3a5fe7227 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:51.135922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b899d-f620-49fe-943b-aea3a5fe7227 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:51.136141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b899d-f620-49fe-943b-aea3a5fe7227 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:51.137607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 494/1002] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:51.142799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ee74d-e729-4925-ac52-eb4ddd13461c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:51.143244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ee74d-e729-4925-ac52-eb4ddd13461c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:51.143333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ee74d-e729-4925-ac52-eb4ddd13461c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:51.161633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0f4ee74d-e729-4925-ac52-eb4ddd13461c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:51.162829 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0f4ee74d-e729-4925-ac52-eb4ddd13461c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:51.211892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:51.232726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 509/1003] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:51.255188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b899d-f620-49fe-943b-aea3a5fe7227 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:51.310460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6c121-d216-4944-bf7d-a3234d3e4a04 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:51.310822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6c121-d216-4944-bf7d-a3234d3e4a04 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:51.310995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6c121-d216-4944-bf7d-a3234d3e4a04 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:51.312529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 510/1004] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:41:51.317065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2472690-bd5c-4ec4-ac43-72f8d5a04ad3 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:51.317400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2472690-bd5c-4ec4-ac43-72f8d5a04ad3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:51.317674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2472690-bd5c-4ec4-ac43-72f8d5a04ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:51.357928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b899d-f620-49fe-943b-aea3a5fe7227 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:51.385852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b2472690-bd5c-4ec4-ac43-72f8d5a04ad3 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:51.386964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b2472690-bd5c-4ec4-ac43-72f8d5a04ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:51.406557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58a8ba-f75a-4fe4-a3b5-e6270a285945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:51.407245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58a8ba-f75a-4fe4-a3b5-e6270a285945 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:51.407535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58a8ba-f75a-4fe4-a3b5-e6270a285945 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:51.407803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58a8ba-f75a-4fe4-a3b5-e6270a285945 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:51.408389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58a8ba-f75a-4fe4-a3b5-e6270a285945 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:51.409741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58a8ba-f75a-4fe4-a3b5-e6270a285945 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-0c58a8ba-f75a-4fe4-a3b5-e6270a285945 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:51.419165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:51.420131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:51.421379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 495/1005] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Aug 8 16:41:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:51.426098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58a8ba-f75a-4fe4-a3b5-e6270a285945 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:51.460101 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 511/1006] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:51.472908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:51.483078 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2472690-bd5c-4ec4-ac43-72f8d5a04ad3 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:51.528364 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58a8ba-f75a-4fe4-a3b5-e6270a285945 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:51.562959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36440fa7-4253-4c00-8b2f-aa84376c0066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:51.563559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36440fa7-4253-4c00-8b2f-aa84376c0066 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:51.563833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36440fa7-4253-4c00-8b2f-aa84376c0066 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:51.564100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36440fa7-4253-4c00-8b2f-aa84376c0066 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:51.564722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36440fa7-4253-4c00-8b2f-aa84376c0066 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:51.565022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36440fa7-4253-4c00-8b2f-aa84376c0066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:51.566497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36440fa7-4253-4c00-8b2f-aa84376c0066 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:51.571521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 496/1007] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:51.577021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36440fa7-4253-4c00-8b2f-aa84376c0066 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:51.611556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2472690-bd5c-4ec4-ac43-72f8d5a04ad3 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:51.623652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:51.649708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a2dfd-594d-4d9a-b80a-b527b0f8493a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:51.650283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a2dfd-594d-4d9a-b80a-b527b0f8493a None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:51.650566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a2dfd-594d-4d9a-b80a-b527b0f8493a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:51.650843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a2dfd-594d-4d9a-b80a-b527b0f8493a None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:51.651505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a2dfd-594d-4d9a-b80a-b527b0f8493a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:51.652970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a2dfd-594d-4d9a-b80a-b527b0f8493a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:51.686895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36440fa7-4253-4c00-8b2f-aa84376c0066 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:51.697275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:51.724007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba88d2-16d6-4b17-859d-588dcc5044c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:51.724754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba88d2-16d6-4b17-859d-588dcc5044c9 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:51.724930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba88d2-16d6-4b17-859d-588dcc5044c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:51.725163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba88d2-16d6-4b17-859d-588dcc5044c9 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:51.726115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba88d2-16d6-4b17-859d-588dcc5044c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:51.726220 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba88d2-16d6-4b17-859d-588dcc5044c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:51.727702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba88d2-16d6-4b17-859d-588dcc5044c9 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:51.731749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 512/1008] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:41:51 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:41:51.737597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 497/1009] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Aug 8 16:41:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:51.738940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a2dfd-594d-4d9a-b80a-b527b0f8493a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:51.855398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a2dfd-594d-4d9a-b80a-b527b0f8493a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:51.893009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b3e1c-00ce-4dc4-b7c8-eda1f553f21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:51.893620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b3e1c-00ce-4dc4-b7c8-eda1f553f21a None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:51.893992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b3e1c-00ce-4dc4-b7c8-eda1f553f21a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:51.894260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b3e1c-00ce-4dc4-b7c8-eda1f553f21a None admin] PATH_INFO: `/v3/endpoints` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:51.894911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b3e1c-00ce-4dc4-b7c8-eda1f553f21a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:51.896886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b3e1c-00ce-4dc4-b7c8-eda1f553f21a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:51.960389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 513/1010] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:41:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:52.561342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a155eb-4cfa-4a7d-b323-bc5fc67a7e74 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:52.561919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a155eb-4cfa-4a7d-b323-bc5fc67a7e74 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:52.561919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a155eb-4cfa-4a7d-b323-bc5fc67a7e74 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:52.563411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 498/1011] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:52.568310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630bda7-e704-4f03-a614-163557180e32 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:52.568697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630bda7-e704-4f03-a614-163557180e32 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:52.568857 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630bda7-e704-4f03-a614-163557180e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:52.587198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b630bda7-e704-4f03-a614-163557180e32 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:52.588394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b630bda7-e704-4f03-a614-163557180e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:52.610638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:52.662748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 514/1012] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:52 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:52.686030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a155eb-4cfa-4a7d-b323-bc5fc67a7e74 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:52.788392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a155eb-4cfa-4a7d-b323-bc5fc67a7e74 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:52.822192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9448e-459d-4a60-bb47-754024aecb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:52.823106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9448e-459d-4a60-bb47-754024aecb1f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:52.823586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9448e-459d-4a60-bb47-754024aecb1f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:52.824075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9448e-459d-4a60-bb47-754024aecb1f None admin] PATH_INFO: `/v3/services/compute` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:52.824866 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9448e-459d-4a60-bb47-754024aecb1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:52.826463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9448e-459d-4a60-bb47-754024aecb1f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-c1d9448e-459d-4a60-bb47-754024aecb1f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:52.835823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:52.836821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 16:41:52.837655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:52.839007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 499/1013] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:52.845316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630bda7-e704-4f03-a614-163557180e32 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:52.945853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630bda7-e704-4f03-a614-163557180e32 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:52.979290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f056c89-dbcd-4b83-a2da-8818ba570b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:52.979893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f056c89-dbcd-4b83-a2da-8818ba570b0a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:52.980195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f056c89-dbcd-4b83-a2da-8818ba570b0a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:52.980466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f056c89-dbcd-4b83-a2da-8818ba570b0a None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:52.981092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f056c89-dbcd-4b83-a2da-8818ba570b0a None admin] RBAC: Adding query filter params (name=compute) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:52.981391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f056c89-dbcd-4b83-a2da-8818ba570b0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:52.982777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f056c89-dbcd-4b83-a2da-8818ba570b0a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:52.992788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 515/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:52.994023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9448e-459d-4a60-bb47-754024aecb1f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:53.030821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:53.099959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9448e-459d-4a60-bb47-754024aecb1f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:53.134832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bbbcb-1c07-44e1-be6b-f0e47cd0cad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:53.135431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bbbcb-1c07-44e1-be6b-f0e47cd0cad8 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:53.135703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bbbcb-1c07-44e1-be6b-f0e47cd0cad8 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:53.135966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bbbcb-1c07-44e1-be6b-f0e47cd0cad8 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:53.136578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bbbcb-1c07-44e1-be6b-f0e47cd0cad8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:53.136874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bbbcb-1c07-44e1-be6b-f0e47cd0cad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:53.138244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bbbcb-1c07-44e1-be6b-f0e47cd0cad8 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:53.147791 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 500/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Aug 8 16:41:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:53.149342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f056c89-dbcd-4b83-a2da-8818ba570b0a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:53.168897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:53.251206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f056c89-dbcd-4b83-a2da-8818ba570b0a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:53.283748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2764698-4003-496f-a5d7-04777846d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:53.284342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2764698-4003-496f-a5d7-04777846d0d0 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:53.284613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2764698-4003-496f-a5d7-04777846d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:53.284875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2764698-4003-496f-a5d7-04777846d0d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:53.285519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2764698-4003-496f-a5d7-04777846d0d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:53.287417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2764698-4003-496f-a5d7-04777846d0d0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:53.405898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 516/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:41:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:53.762415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d25b0f-ebb5-44d2-800a-2dc1681ebf54 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:53.762956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d25b0f-ebb5-44d2-800a-2dc1681ebf54 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:53.762956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d25b0f-ebb5-44d2-800a-2dc1681ebf54 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:53.764314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 501/1017] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:53.769368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ef5ee-b747-4e5c-9af0-44d3095a3ecd None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:53.769765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ef5ee-b747-4e5c-9af0-44d3095a3ecd None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:53.769961 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ef5ee-b747-4e5c-9af0-44d3095a3ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:53.788753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c73ef5ee-b747-4e5c-9af0-44d3095a3ecd None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:53.789957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c73ef5ee-b747-4e5c-9af0-44d3095a3ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:53.856638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:53.863884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 517/1018] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:53.886381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d25b0f-ebb5-44d2-800a-2dc1681ebf54 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:53.989732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:53.991443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43809e93-e140-4455-946e-44a08224fd30 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:53.991725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43809e93-e140-4455-946e-44a08224fd30 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:53.991997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43809e93-e140-4455-946e-44a08224fd30 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:53.993737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 518/1019] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:53.997959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d25b0f-ebb5-44d2-800a-2dc1681ebf54 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:54.009955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0fd40-154c-48e9-84f9-3ad5e84fd12d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:54.009955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0fd40-154c-48e9-84f9-3ad5e84fd12d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:54.009955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0fd40-154c-48e9-84f9-3ad5e84fd12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:54.017020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-84f0fd40-154c-48e9-84f9-3ad5e84fd12d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:54.018206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-84f0fd40-154c-48e9-84f9-3ad5e84fd12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:54.032941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ac58c-932e-4ddc-aa46-4cdd88b5fc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:54.033569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ac58c-932e-4ddc-aa46-4cdd88b5fc5a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:54.033992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ac58c-932e-4ddc-aa46-4cdd88b5fc5a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:54.034264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ac58c-932e-4ddc-aa46-4cdd88b5fc5a None admin] PATH_INFO: `/v3/services/image` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:54.034952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ac58c-932e-4ddc-aa46-4cdd88b5fc5a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:54.037052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ac58c-932e-4ddc-aa46-4cdd88b5fc5a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-264ac58c-932e-4ddc-aa46-4cdd88b5fc5a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:54.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:54.047194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:54.048036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 502/1020] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Aug 8 16:41:53 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:54.054036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ac58c-932e-4ddc-aa46-4cdd88b5fc5a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:54.089836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 519/1021] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:54.111553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0fd40-154c-48e9-84f9-3ad5e84fd12d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:54.160593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ac58c-932e-4ddc-aa46-4cdd88b5fc5a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:54.198248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a0d63-252b-49fd-a650-16a7736cdf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:54.198857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a0d63-252b-49fd-a650-16a7736cdf5e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:54.199135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a0d63-252b-49fd-a650-16a7736cdf5e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:54.199403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a0d63-252b-49fd-a650-16a7736cdf5e None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:54.200033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a0d63-252b-49fd-a650-16a7736cdf5e None admin] RBAC: Adding query filter params (name=image) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:54.200334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a0d63-252b-49fd-a650-16a7736cdf5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:54.201691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a0d63-252b-49fd-a650-16a7736cdf5e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:54.206665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 503/1022] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:54.213768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a0d63-252b-49fd-a650-16a7736cdf5e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:54.227903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0fd40-154c-48e9-84f9-3ad5e84fd12d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:54.261385 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:54.268787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:54.280547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6a0ea-7be3-46ac-8b81-c731ed758868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:54.281529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6a0ea-7be3-46ac-8b81-c731ed758868 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:54.282036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6a0ea-7be3-46ac-8b81-c731ed758868 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:54.282487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6a0ea-7be3-46ac-8b81-c731ed758868 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:54.284175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6a0ea-7be3-46ac-8b81-c731ed758868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:54.287143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6a0ea-7be3-46ac-8b81-c731ed758868 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:54.290067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 520/1023] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:54.295485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6a0ea-7be3-46ac-8b81-c731ed758868 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:54.352102 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a0d63-252b-49fd-a650-16a7736cdf5e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:54.388370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c7966-0c7c-4518-84df-60a328738aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:54.388973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c7966-0c7c-4518-84df-60a328738aa5 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:54.389247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c7966-0c7c-4518-84df-60a328738aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:54.389512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c7966-0c7c-4518-84df-60a328738aa5 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:54.390219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c7966-0c7c-4518-84df-60a328738aa5 None admin] RBAC: Adding query filter params (type=image) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:54.390518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c7966-0c7c-4518-84df-60a328738aa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:54.391913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c7966-0c7c-4518-84df-60a328738aa5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:54.397537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 504/1024] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:54.403029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c7966-0c7c-4518-84df-60a328738aa5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:54.406590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6a0ea-7be3-46ac-8b81-c731ed758868 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:54.445467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac15b0-36c3-4b50-8a78-019b6c090a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:54.446103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac15b0-36c3-4b50-8a78-019b6c090a65 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:54.446382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac15b0-36c3-4b50-8a78-019b6c090a65 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:54.446650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac15b0-36c3-4b50-8a78-019b6c090a65 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:54.447280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac15b0-36c3-4b50-8a78-019b6c090a65 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:54.448704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac15b0-36c3-4b50-8a78-019b6c090a65 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:54.499286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:54.508257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c7966-0c7c-4518-84df-60a328738aa5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:54.533901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 521/1025] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:41:54 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:54.543146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff63b3f-3871-4ef2-b0bc-75c82ca4f124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:54.543817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63b3f-3871-4ef2-b0bc-75c82ca4f124 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:54.544093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63b3f-3871-4ef2-b0bc-75c82ca4f124 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:54.544363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff63b3f-3871-4ef2-b0bc-75c82ca4f124 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:54.545085 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff63b3f-3871-4ef2-b0bc-75c82ca4f124 None admin] RBAC: Adding query filter params (interface=public, service_id=7ce45de8f8d144288a813ccf10e25972, region_id=RegionOne) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:54.545442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff63b3f-3871-4ef2-b0bc-75c82ca4f124 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:54.546890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff63b3f-3871-4ef2-b0bc-75c82ca4f124 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:54.553707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 505/1026] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Aug 8 16:41:54 2022] GET /identity/v3/endpoints?service_id=7ce45de8f8d144288a813ccf10e25972&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:54.559227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac15b0-36c3-4b50-8a78-019b6c090a65 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:54.624090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:54.679321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac15b0-36c3-4b50-8a78-019b6c090a65 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:54.717420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220346c4-88a8-4362-8796-cb472cd69449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:54.718051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220346c4-88a8-4362-8796-cb472cd69449 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:54.718327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220346c4-88a8-4362-8796-cb472cd69449 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:54.718589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220346c4-88a8-4362-8796-cb472cd69449 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:54.719185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220346c4-88a8-4362-8796-cb472cd69449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:54.720537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220346c4-88a8-4362-8796-cb472cd69449 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:54.727396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 522/1027] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Aug 8 16:41:54 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:41:55.510024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d3f3d-a9a1-4cfc-990a-539e6cf96bec None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:55.510548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d3f3d-a9a1-4cfc-990a-539e6cf96bec None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:55.510548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d3f3d-a9a1-4cfc-990a-539e6cf96bec None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:55.511922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 506/1028] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:55.517008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bc6e0-407b-42ae-97ee-6bb03f4f1ab4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:55.517431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bc6e0-407b-42ae-97ee-6bb03f4f1ab4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:55.517599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bc6e0-407b-42ae-97ee-6bb03f4f1ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:55.535365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dc4bc6e0-407b-42ae-97ee-6bb03f4f1ab4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:55.537052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dc4bc6e0-407b-42ae-97ee-6bb03f4f1ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:55.606967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 523/1029] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:55.629957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d3f3d-a9a1-4cfc-990a-539e6cf96bec None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:55.737429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d3f3d-a9a1-4cfc-990a-539e6cf96bec None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:55.773512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4689567-0754-427c-bd98-cd708383d57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:55.774097 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4689567-0754-427c-bd98-cd708383d57a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:55.774368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4689567-0754-427c-bd98-cd708383d57a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:55.774633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4689567-0754-427c-bd98-cd708383d57a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:55.785629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4689567-0754-427c-bd98-cd708383d57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a4689567-0754-427c-bd98-cd708383d57a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:55.790798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:55.791929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:55.792974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 507/1030] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Aug 8 16:41:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:55.799865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bc6e0-407b-42ae-97ee-6bb03f4f1ab4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:55.845902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:55.902491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bc6e0-407b-42ae-97ee-6bb03f4f1ab4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:55.938168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5aaa9b-16fc-450c-a9ff-69f9a52b2ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:55.938759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aaa9b-16fc-450c-a9ff-69f9a52b2ef2 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:55.939039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aaa9b-16fc-450c-a9ff-69f9a52b2ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:55.939292 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aaa9b-16fc-450c-a9ff-69f9a52b2ef2 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:55.939943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5aaa9b-16fc-450c-a9ff-69f9a52b2ef2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:55.940292 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5aaa9b-16fc-450c-a9ff-69f9a52b2ef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:55.941646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5aaa9b-16fc-450c-a9ff-69f9a52b2ef2 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:55.950063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 524/1031] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Aug 8 16:41:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:55.955484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4689567-0754-427c-bd98-cd708383d57a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:55.975862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:56.056084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4689567-0754-427c-bd98-cd708383d57a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:56.089396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535603ec-55be-426d-ba99-e03b54851030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:56.090035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535603ec-55be-426d-ba99-e03b54851030 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.090310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535603ec-55be-426d-ba99-e03b54851030 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.090581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535603ec-55be-426d-ba99-e03b54851030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.101109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535603ec-55be-426d-ba99-e03b54851030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:56.102573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535603ec-55be-426d-ba99-e03b54851030 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-535603ec-55be-426d-ba99-e03b54851030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:56.116741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:56.117693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:56.118640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:56.118640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:56.118640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:56.122109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 508/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:56.123517 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5aaa9b-16fc-450c-a9ff-69f9a52b2ef2 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:56.223021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5aaa9b-16fc-450c-a9ff-69f9a52b2ef2 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:56.225465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:56.257342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14bb5b-055c-4c9a-9833-ee53ad3ff466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:56.257975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14bb5b-055c-4c9a-9833-ee53ad3ff466 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.258252 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14bb5b-055c-4c9a-9833-ee53ad3ff466 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.258515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14bb5b-055c-4c9a-9833-ee53ad3ff466 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.259141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14bb5b-055c-4c9a-9833-ee53ad3ff466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:56.259459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14bb5b-055c-4c9a-9833-ee53ad3ff466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:56.261165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14bb5b-055c-4c9a-9833-ee53ad3ff466 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:56.278011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 525/1033] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:56.279196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535603ec-55be-426d-ba99-e03b54851030 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:56.348702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:56.382730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535603ec-55be-426d-ba99-e03b54851030 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:56.417255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcbea5-576b-4a05-a5ba-f3759978aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:56.417903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcbea5-576b-4a05-a5ba-f3759978aef3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.418176 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcbea5-576b-4a05-a5ba-f3759978aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.418439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcbea5-576b-4a05-a5ba-f3759978aef3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.425809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcbea5-576b-4a05-a5ba-f3759978aef3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:56.427337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcbea5-576b-4a05-a5ba-f3759978aef3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a6bcbea5-576b-4a05-a5ba-f3759978aef3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:41:56.438351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:56.439319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:56.445781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 509/1034] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Aug 8 16:41:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:56.446424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14bb5b-055c-4c9a-9833-ee53ad3ff466 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:56.545413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14bb5b-055c-4c9a-9833-ee53ad3ff466 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:56.579365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067992c-6c19-4651-a733-0e2490ba8442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:56.579962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067992c-6c19-4651-a733-0e2490ba8442 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.580232 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067992c-6c19-4651-a733-0e2490ba8442 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.580496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067992c-6c19-4651-a733-0e2490ba8442 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.581215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067992c-6c19-4651-a733-0e2490ba8442 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:56.581515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067992c-6c19-4651-a733-0e2490ba8442 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:56.585303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067992c-6c19-4651-a733-0e2490ba8442 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:56.602844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfeee3-63f0-4d23-b3a9-3535e6c29b96 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.603312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfeee3-63f0-4d23-b3a9-3535e6c29b96 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.603312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfeee3-63f0-4d23-b3a9-3535e6c29b96 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.603717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 526/1035] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Aug 8 16:41:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:56.604725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 510/1036] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:41:56.608574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067992c-6c19-4651-a733-0e2490ba8442 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:56.609386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ffb1f-4cac-4882-a3a4-d11b739aff9e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.609666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ffb1f-4cac-4882-a3a4-d11b739aff9e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.609966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ffb1f-4cac-4882-a3a4-d11b739aff9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.628969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-2e8ffb1f-4cac-4882-a3a4-d11b739aff9e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:56.630764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2e8ffb1f-4cac-4882-a3a4-d11b739aff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:56.635600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:56.704566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 511/1037] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:56.717977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067992c-6c19-4651-a733-0e2490ba8442 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:56.722620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368f191-5766-4ebe-8ec8-faf8af588024 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.722906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368f191-5766-4ebe-8ec8-faf8af588024 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.723251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368f191-5766-4ebe-8ec8-faf8af588024 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.724709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 512/1038] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:41:56.728315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368f191-5766-4ebe-8ec8-faf8af588024 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:56.755433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08604f25-ac36-4619-9972-2079cf6ca6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:56.756039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08604f25-ac36-4619-9972-2079cf6ca6da None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.756319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08604f25-ac36-4619-9972-2079cf6ca6da None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.756590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08604f25-ac36-4619-9972-2079cf6ca6da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.786607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:56.825171 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08604f25-ac36-4619-9972-2079cf6ca6da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:56.826646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08604f25-ac36-4619-9972-2079cf6ca6da None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:56.831355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368f191-5766-4ebe-8ec8-faf8af588024 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:56.849803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:56.867046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768d91c-fec9-409e-8ca9-8b0c84151173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:56.867598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768d91c-fec9-409e-8ca9-8b0c84151173 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.867873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768d91c-fec9-409e-8ca9-8b0c84151173 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.868140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768d91c-fec9-409e-8ca9-8b0c84151173 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.869229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768d91c-fec9-409e-8ca9-8b0c84151173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:56.870684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768d91c-fec9-409e-8ca9-8b0c84151173 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:56.872432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 513/1039] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:56.873372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357bc4f-2509-4a6e-a158-3fd45a512e62 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:56.873674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357bc4f-2509-4a6e-a158-3fd45a512e62 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:56.873955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357bc4f-2509-4a6e-a158-3fd45a512e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:56.891825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-9357bc4f-2509-4a6e-a158-3fd45a512e62 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:56.892993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9357bc4f-2509-4a6e-a158-3fd45a512e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:56.929742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 527/1040] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:56 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:41:56.930589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08604f25-ac36-4619-9972-2079cf6ca6da None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:56.961590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 514/1041] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:56.962452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357bc4f-2509-4a6e-a158-3fd45a512e62 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:57.017830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:57.032919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08604f25-ac36-4619-9972-2079cf6ca6da None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.055406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:57.060519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357bc4f-2509-4a6e-a158-3fd45a512e62 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.066708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6980ab-9010-4f37-b6d6-067a482f0c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.067316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6980ab-9010-4f37-b6d6-067a482f0c47 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.067609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6980ab-9010-4f37-b6d6-067a482f0c47 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.067868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6980ab-9010-4f37-b6d6-067a482f0c47 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.068505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6980ab-9010-4f37-b6d6-067a482f0c47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.070238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6980ab-9010-4f37-b6d6-067a482f0c47 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.094187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96548c49-238e-499b-82fa-6e12273ff3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.094806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96548c49-238e-499b-82fa-6e12273ff3d0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.095082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96548c49-238e-499b-82fa-6e12273ff3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.095352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96548c49-238e-499b-82fa-6e12273ff3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.105585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96548c49-238e-499b-82fa-6e12273ff3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.107030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96548c49-238e-499b-82fa-6e12273ff3d0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-96548c49-238e-499b-82fa-6e12273ff3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:41:57.121595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:57.122667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:41:57.123696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:41:57.123696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:41:57.123696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:57.123696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 515/1042] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:57.123696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96548c49-238e-499b-82fa-6e12273ff3d0 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:57.210571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 528/1043] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:41:56 2022] POST /identity/v3/users => generated 285 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:41:57.211231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6980ab-9010-4f37-b6d6-067a482f0c47 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:57.230182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96548c49-238e-499b-82fa-6e12273ff3d0 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.241092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:57.275169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846f607-0a02-4b61-8115-a0b0ea097205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.276249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846f607-0a02-4b61-8115-a0b0ea097205 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.276792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846f607-0a02-4b61-8115-a0b0ea097205 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.277120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846f607-0a02-4b61-8115-a0b0ea097205 None admin] PATH_INFO: `/v3/services/image` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.277700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846f607-0a02-4b61-8115-a0b0ea097205 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.280223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846f607-0a02-4b61-8115-a0b0ea097205 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-1846f607-0a02-4b61-8115-a0b0ea097205 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:57.290812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:57.292071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:57.292896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 516/1044] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Aug 8 16:41:57 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:57.298777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846f607-0a02-4b61-8115-a0b0ea097205 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:57.344878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6980ab-9010-4f37-b6d6-067a482f0c47 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.387196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3a666-c822-4be2-98b0-e7fcf5e3c341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.387196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3a666-c822-4be2-98b0-e7fcf5e3c341 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.387196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3a666-c822-4be2-98b0-e7fcf5e3c341 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.387711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3a666-c822-4be2-98b0-e7fcf5e3c341 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.388359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3a666-c822-4be2-98b0-e7fcf5e3c341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:57.388873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3a666-c822-4be2-98b0-e7fcf5e3c341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.390040 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3a666-c822-4be2-98b0-e7fcf5e3c341 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.405272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 529/1045] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:57.411302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3a666-c822-4be2-98b0-e7fcf5e3c341 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:57.414534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846f607-0a02-4b61-8115-a0b0ea097205 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.455803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4a22e-4160-41d4-8683-a049a84348f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.456414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4a22e-4160-41d4-8683-a049a84348f8 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.456688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4a22e-4160-41d4-8683-a049a84348f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.456958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4a22e-4160-41d4-8683-a049a84348f8 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.458165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4a22e-4160-41d4-8683-a049a84348f8 None admin] RBAC: Adding query filter params (name=image) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:57.458557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4a22e-4160-41d4-8683-a049a84348f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.459119 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:57.460442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4a22e-4160-41d4-8683-a049a84348f8 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.466810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 517/1046] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:57.472399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4a22e-4160-41d4-8683-a049a84348f8 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:57.509054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:57.527335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3a666-c822-4be2-98b0-e7fcf5e3c341 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.563489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc06ca-3bbd-469e-9639-2f72b0f489d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.564189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc06ca-3bbd-469e-9639-2f72b0f489d0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.564442 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc06ca-3bbd-469e-9639-2f72b0f489d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.564716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc06ca-3bbd-469e-9639-2f72b0f489d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.571674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc06ca-3bbd-469e-9639-2f72b0f489d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.573112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc06ca-3bbd-469e-9639-2f72b0f489d0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.576013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4a22e-4160-41d4-8683-a049a84348f8 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-53bc06ca-3bbd-469e-9639-2f72b0f489d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:41:57.585430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:57.586516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:57.587385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:41:57.587385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:41:57.587385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:41:57.587385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:41:57.587385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:41:57.587385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:41:57.587385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:41:57.587385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:41:57.591444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 530/1047] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:41:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:57.592317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc06ca-3bbd-469e-9639-2f72b0f489d0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:57.620151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b05df-ab4f-4209-aef9-3310ed88f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.620870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b05df-ab4f-4209-aef9-3310ed88f629 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.621150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b05df-ab4f-4209-aef9-3310ed88f629 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.621419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b05df-ab4f-4209-aef9-3310ed88f629 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.622080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b05df-ab4f-4209-aef9-3310ed88f629 None admin] RBAC: Adding query filter params (type=image) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:57.622382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b05df-ab4f-4209-aef9-3310ed88f629 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.623737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b05df-ab4f-4209-aef9-3310ed88f629 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.629861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 518/1048] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:41:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:57.635176 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b05df-ab4f-4209-aef9-3310ed88f629 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:57.698743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:57.702715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc06ca-3bbd-469e-9639-2f72b0f489d0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.737978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e39ff9-5f90-4a5f-8392-f5d1659afce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.738475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:57.738611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e39ff9-5f90-4a5f-8392-f5d1659afce4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.738962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e39ff9-5f90-4a5f-8392-f5d1659afce4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.739329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e39ff9-5f90-4a5f-8392-f5d1659afce4 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.740369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e39ff9-5f90-4a5f-8392-f5d1659afce4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:57.741063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e39ff9-5f90-4a5f-8392-f5d1659afce4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.742034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b05df-ab4f-4209-aef9-3310ed88f629 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.743827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e39ff9-5f90-4a5f-8392-f5d1659afce4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.756552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 531/1049] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:41:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:57.762551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e39ff9-5f90-4a5f-8392-f5d1659afce4 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:57.785020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9600b-c483-4745-9cf9-b945fe69b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.785816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9600b-c483-4745-9cf9-b945fe69b8aa None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.786099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9600b-c483-4745-9cf9-b945fe69b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.786270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9600b-c483-4745-9cf9-b945fe69b8aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.786911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9600b-c483-4745-9cf9-b945fe69b8aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.788261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9600b-c483-4745-9cf9-b945fe69b8aa None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.856473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 519/1050] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:41:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:41:57.877579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e39ff9-5f90-4a5f-8392-f5d1659afce4 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:57.922690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb4a6b-cae5-4d0d-b565-f94e02bf045a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:57.923301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb4a6b-cae5-4d0d-b565-f94e02bf045a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:57.923575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb4a6b-cae5-4d0d-b565-f94e02bf045a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:57.923844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb4a6b-cae5-4d0d-b565-f94e02bf045a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:57.924593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb4a6b-cae5-4d0d-b565-f94e02bf045a None admin] RBAC: Adding query filter params (role.id=3455a742ed0a4787a4e60d7f104a586a, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=54b05cbc41b844d8932ba4c15d960289) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:57.924887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb4a6b-cae5-4d0d-b565-f94e02bf045a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:57.926422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb4a6b-cae5-4d0d-b565-f94e02bf045a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:57.942706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 532/1051] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:41:57 2022] GET /identity/v3/role_assignments?user.id=54b05cbc41b844d8932ba4c15d960289&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=3455a742ed0a4787a4e60d7f104a586a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:59.279551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631c8c9-01b5-44c0-8351-abab19919c0f None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:59.280073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631c8c9-01b5-44c0-8351-abab19919c0f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:59.280073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631c8c9-01b5-44c0-8351-abab19919c0f None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:59.281603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 520/1052] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:59.287372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d937ee-70f8-408c-be16-d2dfd2b670c5 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:59.287766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d937ee-70f8-408c-be16-d2dfd2b670c5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:59.287924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d937ee-70f8-408c-be16-d2dfd2b670c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:59.306648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a0d937ee-70f8-408c-be16-d2dfd2b670c5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:59.307827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a0d937ee-70f8-408c-be16-d2dfd2b670c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:59.321625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:59.375859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 533/1053] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:41:59.399048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631c8c9-01b5-44c0-8351-abab19919c0f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:59.502582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631c8c9-01b5-44c0-8351-abab19919c0f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:59.536706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c68d3f-2d19-47ad-bd6b-9d374f17fc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:59.537247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c68d3f-2d19-47ad-bd6b-9d374f17fc42 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:59.537520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c68d3f-2d19-47ad-bd6b-9d374f17fc42 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:59.537816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c68d3f-2d19-47ad-bd6b-9d374f17fc42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:59.548266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c68d3f-2d19-47ad-bd6b-9d374f17fc42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-70c68d3f-2d19-47ad-bd6b-9d374f17fc42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:41:59.552918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:41:59.553843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:41:59.554773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:41:59.556204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 521/1054] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:41:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:41:59.560063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d937ee-70f8-408c-be16-d2dfd2b670c5 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:59.668822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d937ee-70f8-408c-be16-d2dfd2b670c5 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:59.702861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f432fbd-9e1e-458e-9f1b-94fb9033f08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:59.703454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f432fbd-9e1e-458e-9f1b-94fb9033f08b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:59.703739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f432fbd-9e1e-458e-9f1b-94fb9033f08b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:59.703991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f432fbd-9e1e-458e-9f1b-94fb9033f08b None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:59.704639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f432fbd-9e1e-458e-9f1b-94fb9033f08b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:41:59.704932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f432fbd-9e1e-458e-9f1b-94fb9033f08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:59.706318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f432fbd-9e1e-458e-9f1b-94fb9033f08b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:59.719725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 534/1055] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:41:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:41:59.721185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c68d3f-2d19-47ad-bd6b-9d374f17fc42 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:59.740408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:59.820340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c68d3f-2d19-47ad-bd6b-9d374f17fc42 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:59.857290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ab7f-1696-45a0-b986-fda5aed540ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:41:59.857940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ab7f-1696-45a0-b986-fda5aed540ca None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:59.858213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ab7f-1696-45a0-b986-fda5aed540ca None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:59.858476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ab7f-1696-45a0-b986-fda5aed540ca None admin] PATH_INFO: `/v3/users/6324fb13cedb48e9ae8d0f955270b25b` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:59.872192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3ab7f-1696-45a0-b986-fda5aed540ca None admin] RBAC: Authorizing `identity:get_user(user_id=6324fb13cedb48e9ae8d0f955270b25b)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:41:59.873676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3ab7f-1696-45a0-b986-fda5aed540ca None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:41:59.880460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 522/1056] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:41:59 2022] GET /identity/v3/users/6324fb13cedb48e9ae8d0f955270b25b => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:41:59.881666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f432fbd-9e1e-458e-9f1b-94fb9033f08b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:41:59.893666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011d75-51d0-4acc-931e-7bc836397db3 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:59.894033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011d75-51d0-4acc-931e-7bc836397db3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:59.894301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011d75-51d0-4acc-931e-7bc836397db3 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:59.895539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:41:59.896651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 523/1057] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:41:59.901405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309407d2-2de3-46b5-a136-f94b8595ce45 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:59.905668 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309407d2-2de3-46b5-a136-f94b8595ce45 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:59.905668 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309407d2-2de3-46b5-a136-f94b8595ce45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:59.919439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-309407d2-2de3-46b5-a136-f94b8595ce45 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:41:59.920636 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-309407d2-2de3-46b5-a136-f94b8595ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:41:59.986638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 524/1058] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:41:59.987870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940bb39-7e19-4150-9c59-31ff22b163e1 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:59.988152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940bb39-7e19-4150-9c59-31ff22b163e1 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:59.988464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940bb39-7e19-4150-9c59-31ff22b163e1 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:41:59.988966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f432fbd-9e1e-458e-9f1b-94fb9033f08b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:41:59.993607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 525/1059] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:41:59.995271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19706d-6941-4cbb-a91d-b87725c11c7d None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:41:59.995604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19706d-6941-4cbb-a91d-b87725c11c7d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:41:59.996377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19706d-6941-4cbb-a91d-b87725c11c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.003716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:00.014925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-da19706d-6941-4cbb-a91d-b87725c11c7d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:00.016095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-da19706d-6941-4cbb-a91d-b87725c11c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:00.023708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22296f-4e36-4c64-876f-110b8e793f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:00.024319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22296f-4e36-4c64-876f-110b8e793f36 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:00.024590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22296f-4e36-4c64-876f-110b8e793f36 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:00.024856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22296f-4e36-4c64-876f-110b8e793f36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.081548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 526/1060] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:41:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:00.083027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19706d-6941-4cbb-a91d-b87725c11c7d None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:00.101152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22296f-4e36-4c64-876f-110b8e793f36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:00.102897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22296f-4e36-4c64-876f-110b8e793f36 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:00.114433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:00.180284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19706d-6941-4cbb-a91d-b87725c11c7d None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:00.199959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 535/1061] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:41:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:42:00.202706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22296f-4e36-4c64-876f-110b8e793f36 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:00.215153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f2122-a9b2-4760-977a-5cf59e14c3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:00.215777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f2122-a9b2-4760-977a-5cf59e14c3db None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:00.216051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f2122-a9b2-4760-977a-5cf59e14c3db None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:00.216316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f2122-a9b2-4760-977a-5cf59e14c3db None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.218126 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:00.227236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f2122-a9b2-4760-977a-5cf59e14c3db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-295f2122-a9b2-4760-977a-5cf59e14c3db None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:00.231851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:00.232775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:00.233619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 527/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Aug 8 16:42:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:00.234386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f2122-a9b2-4760-977a-5cf59e14c3db None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:00.305710 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22296f-4e36-4c64-876f-110b8e793f36 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:00.316068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:00.331225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f2122-a9b2-4760-977a-5cf59e14c3db None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:00.346469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c5e72-5c46-455f-890d-85b158cecbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:00.346469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c5e72-5c46-455f-890d-85b158cecbf9 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:00.346469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c5e72-5c46-455f-890d-85b158cecbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:00.346469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c5e72-5c46-455f-890d-85b158cecbf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.365845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cbb98-a9aa-456f-afe5-9d006152a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:00.366467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cbb98-a9aa-456f-afe5-9d006152a03c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:00.366744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cbb98-a9aa-456f-afe5-9d006152a03c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:00.367037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cbb98-a9aa-456f-afe5-9d006152a03c None admin] PATH_INFO: `/v3/projects/36339ca9f884448a9a08bc8ce77a9b13` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.368174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929cbb98-a9aa-456f-afe5-9d006152a03c None admin] RBAC: Authorizing `identity:get_project(project_id=36339ca9f884448a9a08bc8ce77a9b13)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:00.369601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929cbb98-a9aa-456f-afe5-9d006152a03c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:00.371545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 528/1063] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:00 2022] GET /identity/v3/projects/36339ca9f884448a9a08bc8ce77a9b13 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:00.374379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cbb98-a9aa-456f-afe5-9d006152a03c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:00.408437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c5e72-5c46-455f-890d-85b158cecbf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:00.409906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c5e72-5c46-455f-890d-85b158cecbf9 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:00.432132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:00.468765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cbb98-a9aa-456f-afe5-9d006152a03c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:00.503768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7fb89-1076-45c6-9d32-18df4fa00047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:00.504408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7fb89-1076-45c6-9d32-18df4fa00047 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:00.504682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7fb89-1076-45c6-9d32-18df4fa00047 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:00.505127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7fb89-1076-45c6-9d32-18df4fa00047 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.505811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7fb89-1076-45c6-9d32-18df4fa00047 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:00.506120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7fb89-1076-45c6-9d32-18df4fa00047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:00.507082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 536/1064] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:42:00.507468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7fb89-1076-45c6-9d32-18df4fa00047 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:00.507904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c5e72-5c46-455f-890d-85b158cecbf9 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:00.521389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 529/1065] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Aug 8 16:42:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:00.521389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7fb89-1076-45c6-9d32-18df4fa00047 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:00.525495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:00.615312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c5e72-5c46-455f-890d-85b158cecbf9 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:00.618660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7fb89-1076-45c6-9d32-18df4fa00047 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:00.641965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:00.649443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6290e-ed0a-4e80-8c94-4b8b3c73bea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:00.650073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6290e-ed0a-4e80-8c94-4b8b3c73bea1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:00.668803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6290e-ed0a-4e80-8c94-4b8b3c73bea1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:00.668803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6290e-ed0a-4e80-8c94-4b8b3c73bea1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.668803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6290e-ed0a-4e80-8c94-4b8b3c73bea1 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=36339ca9f884448a9a08bc8ce77a9b13, user.id=6324fb13cedb48e9ae8d0f955270b25b) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:00.668803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6290e-ed0a-4e80-8c94-4b8b3c73bea1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:00.669240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859af21-0a43-4f07-bd53-81dd4be306f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:00.669347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6290e-ed0a-4e80-8c94-4b8b3c73bea1 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:00.669420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859af21-0a43-4f07-bd53-81dd4be306f0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:00.669420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859af21-0a43-4f07-bd53-81dd4be306f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:00.669420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859af21-0a43-4f07-bd53-81dd4be306f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.669611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 537/1066] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:42:00 2022] GET /identity/v3/role_assignments?user.id=6324fb13cedb48e9ae8d0f955270b25b&scope.project.id=36339ca9f884448a9a08bc8ce77a9b13&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:00.669611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6290e-ed0a-4e80-8c94-4b8b3c73bea1 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:00.669735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859af21-0a43-4f07-bd53-81dd4be306f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:00.669735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859af21-0a43-4f07-bd53-81dd4be306f0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-9859af21-0a43-4f07-bd53-81dd4be306f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:42:00.687340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:00.688377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:42:00.689276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:42:00.689276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:00.689276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:00.689276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 530/1067] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:00.695835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859af21-0a43-4f07-bd53-81dd4be306f0 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:00.758692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:00.784947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6290e-ed0a-4e80-8c94-4b8b3c73bea1 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:00.816620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859af21-0a43-4f07-bd53-81dd4be306f0 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:00.823649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ea2f2-a0d5-42c6-8cf4-dfca159cdea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:00.824286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ea2f2-a0d5-42c6-8cf4-dfca159cdea1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:00.824561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ea2f2-a0d5-42c6-8cf4-dfca159cdea1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:00.824836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ea2f2-a0d5-42c6-8cf4-dfca159cdea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.836674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ea2f2-a0d5-42c6-8cf4-dfca159cdea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:00.838297 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ea2f2-a0d5-42c6-8cf4-dfca159cdea1 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-667ea2f2-a0d5-42c6-8cf4-dfca159cdea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:42:00.853637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:00.854677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:42:00.855588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:42:00.855588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:00.855588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:00.855588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 538/1068] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:00.857292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49667c67-e05d-473c-95a6-8d1d96fa81d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:00.857928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49667c67-e05d-473c-95a6-8d1d96fa81d0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:00.858202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49667c67-e05d-473c-95a6-8d1d96fa81d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:00.858466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49667c67-e05d-473c-95a6-8d1d96fa81d0 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:00.859105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49667c67-e05d-473c-95a6-8d1d96fa81d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:00.859648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49667c67-e05d-473c-95a6-8d1d96fa81d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:00.860456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ea2f2-a0d5-42c6-8cf4-dfca159cdea1 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:00.860741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49667c67-e05d-473c-95a6-8d1d96fa81d0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:00.885909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 531/1069] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:42:00.982085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ea2f2-a0d5-42c6-8cf4-dfca159cdea1 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:01.010545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:01.019846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539607-51f7-4164-b97b-fb06263c7d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:01.023791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539607-51f7-4164-b97b-fb06263c7d9a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:01.023791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539607-51f7-4164-b97b-fb06263c7d9a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:01.023791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539607-51f7-4164-b97b-fb06263c7d9a None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:01.023791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39539607-51f7-4164-b97b-fb06263c7d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:01.023791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39539607-51f7-4164-b97b-fb06263c7d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:01.023791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39539607-51f7-4164-b97b-fb06263c7d9a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:01.036006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 539/1070] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:42:01.041716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49667c67-e05d-473c-95a6-8d1d96fa81d0 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:01.079177 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:01.148826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49667c67-e05d-473c-95a6-8d1d96fa81d0 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:01.183851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df25163-f9a4-4e8f-a733-187d84107b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:01.184474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df25163-f9a4-4e8f-a733-187d84107b70 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:01.184745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df25163-f9a4-4e8f-a733-187d84107b70 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:01.185007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df25163-f9a4-4e8f-a733-187d84107b70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:01.192517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df25163-f9a4-4e8f-a733-187d84107b70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:01.193954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df25163-f9a4-4e8f-a733-187d84107b70 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-4df25163-f9a4-4e8f-a733-187d84107b70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:42:01.205284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:01.206423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:42:01.207643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:01.209945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 532/1071] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Aug 8 16:42:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:01.212218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539607-51f7-4164-b97b-fb06263c7d9a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:01.309608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539607-51f7-4164-b97b-fb06263c7d9a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:01.342964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596df118-c244-4089-bdaa-dd4c9a1b0e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:01.343566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596df118-c244-4089-bdaa-dd4c9a1b0e8a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:01.343841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596df118-c244-4089-bdaa-dd4c9a1b0e8a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:01.344109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596df118-c244-4089-bdaa-dd4c9a1b0e8a None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:01.344832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596df118-c244-4089-bdaa-dd4c9a1b0e8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:01.347156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596df118-c244-4089-bdaa-dd4c9a1b0e8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:01.347156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596df118-c244-4089-bdaa-dd4c9a1b0e8a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:01.368853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 540/1072] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Aug 8 16:42:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:01.370027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df25163-f9a4-4e8f-a733-187d84107b70 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:01.460333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:01.465799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df25163-f9a4-4e8f-a733-187d84107b70 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:01.499858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff4ad8-74f5-4e73-bc9d-077fdacdaf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:01.500473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff4ad8-74f5-4e73-bc9d-077fdacdaf70 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:01.500747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff4ad8-74f5-4e73-bc9d-077fdacdaf70 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:01.501014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff4ad8-74f5-4e73-bc9d-077fdacdaf70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:01.510613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff4ad8-74f5-4e73-bc9d-077fdacdaf70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:01.512066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff4ad8-74f5-4e73-bc9d-077fdacdaf70 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-36ff4ad8-74f5-4e73-bc9d-077fdacdaf70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:42:01.525882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:01.526970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:42:01.527859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:42:01.527859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:01.527859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:01.527859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 533/1073] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:42:01.532721 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596df118-c244-4089-bdaa-dd4c9a1b0e8a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:01.577479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:01.631042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596df118-c244-4089-bdaa-dd4c9a1b0e8a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:01.663625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84054a52-69ef-4143-b6d1-9158a3b081d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:01.664295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84054a52-69ef-4143-b6d1-9158a3b081d6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:01.664570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84054a52-69ef-4143-b6d1-9158a3b081d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:01.664838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84054a52-69ef-4143-b6d1-9158a3b081d6 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:01.665469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84054a52-69ef-4143-b6d1-9158a3b081d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:01.665768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84054a52-69ef-4143-b6d1-9158a3b081d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:01.667148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84054a52-69ef-4143-b6d1-9158a3b081d6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:01.678762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 541/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:01.684383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff4ad8-74f5-4e73-bc9d-077fdacdaf70 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:01.783048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff4ad8-74f5-4e73-bc9d-077fdacdaf70 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:01.815338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43e5a1-f9e9-4653-86e7-3baf0f7943c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:01.815953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43e5a1-f9e9-4653-86e7-3baf0f7943c5 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:01.816225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43e5a1-f9e9-4653-86e7-3baf0f7943c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:01.816488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43e5a1-f9e9-4653-86e7-3baf0f7943c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:01.823078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43e5a1-f9e9-4653-86e7-3baf0f7943c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:01.824487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43e5a1-f9e9-4653-86e7-3baf0f7943c5 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-ae43e5a1-f9e9-4653-86e7-3baf0f7943c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:42:01.835627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:01.836682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:01.837591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:42:01.837591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:42:01.837591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:42:01.837591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:42:01.837591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:42:01.837591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:42:01.837591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:42:01.837591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:01.837591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 534/1075] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:01.842595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84054a52-69ef-4143-b6d1-9158a3b081d6 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:01.939186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84054a52-69ef-4143-b6d1-9158a3b081d6 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:01.941421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:01.972328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c2cba-8f37-4777-8d50-7e0ed989553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:01.973523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2cba-8f37-4777-8d50-7e0ed989553c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:01.973820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2cba-8f37-4777-8d50-7e0ed989553c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:01.974097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2cba-8f37-4777-8d50-7e0ed989553c None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:01.974783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c2cba-8f37-4777-8d50-7e0ed989553c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:01.975083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c2cba-8f37-4777-8d50-7e0ed989553c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:01.976610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c2cba-8f37-4777-8d50-7e0ed989553c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:01.988287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 542/1076] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:01.994224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43e5a1-f9e9-4653-86e7-3baf0f7943c5 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:02.005706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:02.091372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43e5a1-f9e9-4653-86e7-3baf0f7943c5 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:02.124144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91edbe-e15e-4ff4-bfed-c77db88dc89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:02.124829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91edbe-e15e-4ff4-bfed-c77db88dc89f None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:02.125103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91edbe-e15e-4ff4-bfed-c77db88dc89f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:02.125367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91edbe-e15e-4ff4-bfed-c77db88dc89f None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/54b05cbc41b844d8932ba4c15d960289/roles/3455a742ed0a4787a4e60d7f104a586a` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:02.127483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91edbe-e15e-4ff4-bfed-c77db88dc89f None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=54b05cbc41b844d8932ba4c15d960289, role_id=3455a742ed0a4787a4e60d7f104a586a)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:02.128924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91edbe-e15e-4ff4-bfed-c77db88dc89f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:02.186357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 535/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:42:01 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/54b05cbc41b844d8932ba4c15d960289/roles/3455a742ed0a4787a4e60d7f104a586a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:42:02.671408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955dc6-22ac-423e-a2b7-65809c7bdf2c None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:02.671919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955dc6-22ac-423e-a2b7-65809c7bdf2c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:02.672009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955dc6-22ac-423e-a2b7-65809c7bdf2c None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:02.673280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 543/1078] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:02.678698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39424a5-c65a-4306-a940-5bdd2eab7415 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:02.679106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39424a5-c65a-4306-a940-5bdd2eab7415 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:02.679254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39424a5-c65a-4306-a940-5bdd2eab7415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:02.697084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f39424a5-c65a-4306-a940-5bdd2eab7415 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:02.698310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f39424a5-c65a-4306-a940-5bdd2eab7415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:02.757956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:02.763219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 536/1079] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:02.786131 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9955dc6-22ac-423e-a2b7-65809c7bdf2c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:02.887045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9955dc6-22ac-423e-a2b7-65809c7bdf2c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:02.920194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4931aff-0ffe-43a4-a564-2ec2c65bba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:02.920825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4931aff-0ffe-43a4-a564-2ec2c65bba3a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:02.921099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4931aff-0ffe-43a4-a564-2ec2c65bba3a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:02.921367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4931aff-0ffe-43a4-a564-2ec2c65bba3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:02.931279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4931aff-0ffe-43a4-a564-2ec2c65bba3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-c4931aff-0ffe-43a4-a564-2ec2c65bba3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:02.935845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:02.936940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:02.938046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:02.942137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 544/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:42:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:02.943647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39424a5-c65a-4306-a940-5bdd2eab7415 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:03.038745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39424a5-c65a-4306-a940-5bdd2eab7415 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:03.072382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83933a-a22e-4e26-9fd2-8550b7fb46ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:03.072978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83933a-a22e-4e26-9fd2-8550b7fb46ff None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:03.073254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83933a-a22e-4e26-9fd2-8550b7fb46ff None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:03.073525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83933a-a22e-4e26-9fd2-8550b7fb46ff None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:03.074215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83933a-a22e-4e26-9fd2-8550b7fb46ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:03.074517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83933a-a22e-4e26-9fd2-8550b7fb46ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:03.075867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83933a-a22e-4e26-9fd2-8550b7fb46ff None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:03.083865 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 537/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:42:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:42:03.089434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4931aff-0ffe-43a4-a564-2ec2c65bba3a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:03.115445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:03.187641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4931aff-0ffe-43a4-a564-2ec2c65bba3a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:03.220496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d4b7b-1e41-43d9-bcb0-9fa119569259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:03.221117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4b7b-1e41-43d9-bcb0-9fa119569259 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:03.221394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4b7b-1e41-43d9-bcb0-9fa119569259 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:03.221662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4b7b-1e41-43d9-bcb0-9fa119569259 None admin] PATH_INFO: `/v3/users/6324fb13cedb48e9ae8d0f955270b25b` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:03.223438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d4b7b-1e41-43d9-bcb0-9fa119569259 None admin] RBAC: Authorizing `identity:get_user(user_id=6324fb13cedb48e9ae8d0f955270b25b)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:03.224865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d4b7b-1e41-43d9-bcb0-9fa119569259 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:03.226472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 545/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:03 2022] GET /identity/v3/users/6324fb13cedb48e9ae8d0f955270b25b => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:03.232129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83933a-a22e-4e26-9fd2-8550b7fb46ff None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:03.293128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:03.335372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83933a-a22e-4e26-9fd2-8550b7fb46ff None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:03.368340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e451a4f-1dc1-4f8c-aade-d061e2fc3563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:03.368963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e451a4f-1dc1-4f8c-aade-d061e2fc3563 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:03.369234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e451a4f-1dc1-4f8c-aade-d061e2fc3563 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:03.369501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e451a4f-1dc1-4f8c-aade-d061e2fc3563 None admin] PATH_INFO: `/v3/projects/36339ca9f884448a9a08bc8ce77a9b13` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:03.370616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e451a4f-1dc1-4f8c-aade-d061e2fc3563 None admin] RBAC: Authorizing `identity:get_project(project_id=36339ca9f884448a9a08bc8ce77a9b13)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:03.372048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e451a4f-1dc1-4f8c-aade-d061e2fc3563 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:03.373660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 538/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:03 2022] GET /identity/v3/projects/36339ca9f884448a9a08bc8ce77a9b13 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:03.379712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d4b7b-1e41-43d9-bcb0-9fa119569259 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:03.466742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:03.475471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d4b7b-1e41-43d9-bcb0-9fa119569259 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:03.507623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae7387-3456-4bcd-9807-ef3365d8529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:03.508261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae7387-3456-4bcd-9807-ef3365d8529b None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:03.508539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae7387-3456-4bcd-9807-ef3365d8529b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:03.508809 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae7387-3456-4bcd-9807-ef3365d8529b None admin] PATH_INFO: `/v3/projects/36339ca9f884448a9a08bc8ce77a9b13/users/6324fb13cedb48e9ae8d0f955270b25b/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:03.510712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae7387-3456-4bcd-9807-ef3365d8529b None admin] RBAC: Authorizing `identity:create_grant(project_id=36339ca9f884448a9a08bc8ce77a9b13, user_id=6324fb13cedb48e9ae8d0f955270b25b, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:03.512130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae7387-3456-4bcd-9807-ef3365d8529b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:03.599012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 546/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:42:03 2022] PUT /identity/v3/projects/36339ca9f884448a9a08bc8ce77a9b13/users/6324fb13cedb48e9ae8d0f955270b25b/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:42:04.183824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ff9d9-eca5-4d80-a169-ce08af42a827 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:04.184349 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ff9d9-eca5-4d80-a169-ce08af42a827 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:04.184349 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ff9d9-eca5-4d80-a169-ce08af42a827 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:04.185699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 539/1085] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:04.190818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48331ff-2115-4779-9357-78e84a0fc1f7 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:04.191194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48331ff-2115-4779-9357-78e84a0fc1f7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:04.191429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48331ff-2115-4779-9357-78e84a0fc1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:04.209854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b48331ff-2115-4779-9357-78e84a0fc1f7 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:04.211037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b48331ff-2115-4779-9357-78e84a0fc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:04.275165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 547/1086] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:04.303547 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ff9d9-eca5-4d80-a169-ce08af42a827 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:04.406999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ff9d9-eca5-4d80-a169-ce08af42a827 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:04.432180 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:04.440056 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d98bd1-831a-4e00-abd1-c9c1f6f693ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:04.440615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d98bd1-831a-4e00-abd1-c9c1f6f693ce None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:04.440889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d98bd1-831a-4e00-abd1-c9c1f6f693ce None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:04.441156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d98bd1-831a-4e00-abd1-c9c1f6f693ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:04.452188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d98bd1-831a-4e00-abd1-c9c1f6f693ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-c7d98bd1-831a-4e00-abd1-c9c1f6f693ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:04.456920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:04.458414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:04.459268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 540/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Aug 8 16:42:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:04.464526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48331ff-2115-4779-9357-78e84a0fc1f7 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:04.541967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:04.559254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48331ff-2115-4779-9357-78e84a0fc1f7 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:04.594866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ca60-bde1-4285-b307-b2f65bb300ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:04.595512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ca60-bde1-4285-b307-b2f65bb300ae None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:04.595789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ca60-bde1-4285-b307-b2f65bb300ae None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:04.596061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ca60-bde1-4285-b307-b2f65bb300ae None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:04.596725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81ca60-bde1-4285-b307-b2f65bb300ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:04.597028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81ca60-bde1-4285-b307-b2f65bb300ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:04.598607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81ca60-bde1-4285-b307-b2f65bb300ae None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:04.606504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 548/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Aug 8 16:42:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:04.612125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d98bd1-831a-4e00-abd1-c9c1f6f693ce None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:04.713351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d98bd1-831a-4e00-abd1-c9c1f6f693ce None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:04.746947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce74ce1-4ff4-43c0-a8c5-4cc7eccc35ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:04.747570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce74ce1-4ff4-43c0-a8c5-4cc7eccc35ec None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:04.747849 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce74ce1-4ff4-43c0-a8c5-4cc7eccc35ec None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:04.748120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce74ce1-4ff4-43c0-a8c5-4cc7eccc35ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:04.759205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce74ce1-4ff4-43c0-a8c5-4cc7eccc35ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:04.760709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce74ce1-4ff4-43c0-a8c5-4cc7eccc35ec None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-9ce74ce1-4ff4-43c0-a8c5-4cc7eccc35ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:42:04.774283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:04.775324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:42:04.776209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:42:04.776209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:04.776209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:04.776209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 541/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:04.781206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ca60-bde1-4285-b307-b2f65bb300ae None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:04.877449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ca60-bde1-4285-b307-b2f65bb300ae None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:04.909463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddabe3-457e-45a3-8ce0-7ca6283beaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:04.910105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddabe3-457e-45a3-8ce0-7ca6283beaba None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:04.910386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddabe3-457e-45a3-8ce0-7ca6283beaba None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:04.910688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddabe3-457e-45a3-8ce0-7ca6283beaba None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:04.911339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddabe3-457e-45a3-8ce0-7ca6283beaba None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:04.911643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddabe3-457e-45a3-8ce0-7ca6283beaba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:04.912997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddabe3-457e-45a3-8ce0-7ca6283beaba None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:04.924634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 549/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:04.930643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce74ce1-4ff4-43c0-a8c5-4cc7eccc35ec None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:04.971717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:05.027796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce74ce1-4ff4-43c0-a8c5-4cc7eccc35ec None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:05.060090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272218a-adb4-4036-8471-75678275bf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:05.060707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272218a-adb4-4036-8471-75678275bf16 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:05.060981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272218a-adb4-4036-8471-75678275bf16 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:05.061249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272218a-adb4-4036-8471-75678275bf16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:05.069300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272218a-adb4-4036-8471-75678275bf16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:05.070745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272218a-adb4-4036-8471-75678275bf16 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-d272218a-adb4-4036-8471-75678275bf16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:42:05.081361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:05.082423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:05.083449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 542/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Aug 8 16:42:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:05.088670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddabe3-457e-45a3-8ce0-7ca6283beaba None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:05.092422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:05.185104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddabe3-457e-45a3-8ce0-7ca6283beaba None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:05.225395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c06ae-9016-4b8f-95df-33fc8e787301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:05.226032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c06ae-9016-4b8f-95df-33fc8e787301 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:05.226205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c06ae-9016-4b8f-95df-33fc8e787301 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:05.226468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c06ae-9016-4b8f-95df-33fc8e787301 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:05.227185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c06ae-9016-4b8f-95df-33fc8e787301 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:05.227483 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c06ae-9016-4b8f-95df-33fc8e787301 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:05.228879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c06ae-9016-4b8f-95df-33fc8e787301 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:05.246495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 550/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Aug 8 16:42:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:05.252092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272218a-adb4-4036-8471-75678275bf16 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:05.347967 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:05.351036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272218a-adb4-4036-8471-75678275bf16 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:05.383696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5cfc6-c320-40b5-a5ea-2d38ac9b25ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:05.384336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5cfc6-c320-40b5-a5ea-2d38ac9b25ea None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:05.384610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5cfc6-c320-40b5-a5ea-2d38ac9b25ea None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:05.384873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5cfc6-c320-40b5-a5ea-2d38ac9b25ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:05.450602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5cfc6-c320-40b5-a5ea-2d38ac9b25ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:05.452041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5cfc6-c320-40b5-a5ea-2d38ac9b25ea None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:05.543467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:05.546886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 543/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 16:42:05.552623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c06ae-9016-4b8f-95df-33fc8e787301 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:05.592570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5ca12-5582-40b0-9c64-ef97efaa7055 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:05.592858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5ca12-5582-40b0-9c64-ef97efaa7055 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:05.593129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5ca12-5582-40b0-9c64-ef97efaa7055 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:05.594628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 544/1094] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:42:05.600063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655062bf-9fd3-4cf5-a12d-9ec7425106ec None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:05.600346 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655062bf-9fd3-4cf5-a12d-9ec7425106ec None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:05.600623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655062bf-9fd3-4cf5-a12d-9ec7425106ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:05.615594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:05.617832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-655062bf-9fd3-4cf5-a12d-9ec7425106ec None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:05.619005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-655062bf-9fd3-4cf5-a12d-9ec7425106ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:05.654928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c06ae-9016-4b8f-95df-33fc8e787301 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:05.688017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527e0f1-7cf6-4a67-be5c-65dde2b7e6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:05.688337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 545/1095] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:42:05.690321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527e0f1-7cf6-4a67-be5c-65dde2b7e6e4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:05.690609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527e0f1-7cf6-4a67-be5c-65dde2b7e6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:05.690904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527e0f1-7cf6-4a67-be5c-65dde2b7e6e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:05.701461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527e0f1-7cf6-4a67-be5c-65dde2b7e6e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:05.702928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527e0f1-7cf6-4a67-be5c-65dde2b7e6e4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:05.711902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655062bf-9fd3-4cf5-a12d-9ec7425106ec None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-e527e0f1-7cf6-4a67-be5c-65dde2b7e6e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:42:05.717114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:05.718116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:42:05.718994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:42:05.718994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:42:05.718994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:05.718994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 551/1096] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:05.724229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527e0f1-7cf6-4a67-be5c-65dde2b7e6e4 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:05.810183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655062bf-9fd3-4cf5-a12d-9ec7425106ec None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:05.821134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527e0f1-7cf6-4a67-be5c-65dde2b7e6e4 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:05.834528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:05.838687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:05.844372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9d7bb-bceb-4abc-8bdb-9af37013d589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:05.844998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9d7bb-bceb-4abc-8bdb-9af37013d589 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:05.845273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9d7bb-bceb-4abc-8bdb-9af37013d589 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:05.850206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9d7bb-bceb-4abc-8bdb-9af37013d589 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:05.854283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822d53c-8b28-4cb4-8f48-06f94735b303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:05.854875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822d53c-8b28-4cb4-8f48-06f94735b303 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:05.855150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822d53c-8b28-4cb4-8f48-06f94735b303 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:05.855416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822d53c-8b28-4cb4-8f48-06f94735b303 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:05.856040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9d7bb-bceb-4abc-8bdb-9af37013d589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:05.856134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822d53c-8b28-4cb4-8f48-06f94735b303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:05.856339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822d53c-8b28-4cb4-8f48-06f94735b303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:05.857692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822d53c-8b28-4cb4-8f48-06f94735b303 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-72f9d7bb-bceb-4abc-8bdb-9af37013d589 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:05.860626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:05.861744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:05.862805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:05.865940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 546/1097] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:42:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:42:05.867712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9d7bb-bceb-4abc-8bdb-9af37013d589 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:05.874257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 552/1098] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:42:05.875214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822d53c-8b28-4cb4-8f48-06f94735b303 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:05.966324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9d7bb-bceb-4abc-8bdb-9af37013d589 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:05.974469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822d53c-8b28-4cb4-8f48-06f94735b303 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:06.001054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf0b7a-3d1f-4e87-aa39-b978e88daf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:06.001664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf0b7a-3d1f-4e87-aa39-b978e88daf54 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:06.001970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf0b7a-3d1f-4e87-aa39-b978e88daf54 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:06.002235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf0b7a-3d1f-4e87-aa39-b978e88daf54 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:06.002891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf0b7a-3d1f-4e87-aa39-b978e88daf54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:06.003191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf0b7a-3d1f-4e87-aa39-b978e88daf54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:06.004757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf0b7a-3d1f-4e87-aa39-b978e88daf54 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:06.008816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639cd60-f83e-4b51-baf2-22056119d388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:06.009428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639cd60-f83e-4b51-baf2-22056119d388 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:06.009703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639cd60-f83e-4b51-baf2-22056119d388 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:06.009999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639cd60-f83e-4b51-baf2-22056119d388 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:06.013057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 547/1099] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:42:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:42:06.016425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639cd60-f83e-4b51-baf2-22056119d388 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:06.018122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639cd60-f83e-4b51-baf2-22056119d388 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:06.018683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf0b7a-3d1f-4e87-aa39-b978e88daf54 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-4639cd60-f83e-4b51-baf2-22056119d388 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:42:06.028934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:06.030035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:06.031269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:42:06.031269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:42:06.031269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:42:06.031269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:42:06.031269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:42:06.031269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:42:06.031269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:42:06.031269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:06.031269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 553/1100] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:42:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:42:06.036098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639cd60-f83e-4b51-baf2-22056119d388 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:06.069086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:06.073154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:06.119006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf0b7a-3d1f-4e87-aa39-b978e88daf54 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:06.132443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639cd60-f83e-4b51-baf2-22056119d388 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:06.152415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b9773-5979-4425-b55a-57420985c4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:06.153031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b9773-5979-4425-b55a-57420985c4e2 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:06.153305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b9773-5979-4425-b55a-57420985c4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:06.153571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b9773-5979-4425-b55a-57420985c4e2 None admin] PATH_INFO: `/v3/users/6324fb13cedb48e9ae8d0f955270b25b` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:06.154705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b9773-5979-4425-b55a-57420985c4e2 None admin] RBAC: Authorizing `identity:get_user(user_id=6324fb13cedb48e9ae8d0f955270b25b)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:06.156111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b9773-5979-4425-b55a-57420985c4e2 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:06.157672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 548/1101] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:06 2022] GET /identity/v3/users/6324fb13cedb48e9ae8d0f955270b25b => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:42:06.163941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b9773-5979-4425-b55a-57420985c4e2 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:06.165374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1f6cc-dc22-4009-a1f0-cdac42a53833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:06.165999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1f6cc-dc22-4009-a1f0-cdac42a53833 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:06.166278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1f6cc-dc22-4009-a1f0-cdac42a53833 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:06.166547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1f6cc-dc22-4009-a1f0-cdac42a53833 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:06.167226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1f6cc-dc22-4009-a1f0-cdac42a53833 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:06.167526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1f6cc-dc22-4009-a1f0-cdac42a53833 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:06.168929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1f6cc-dc22-4009-a1f0-cdac42a53833 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:06.180877 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 554/1102] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:42:06.186684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1f6cc-dc22-4009-a1f0-cdac42a53833 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:06.269033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b9773-5979-4425-b55a-57420985c4e2 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:06.275974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:06.282404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1f6cc-dc22-4009-a1f0-cdac42a53833 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:06.289705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:06.303198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c6afc-ea89-487c-882b-2705ff70ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:06.303799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c6afc-ea89-487c-882b-2705ff70ae8c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:06.304072 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c6afc-ea89-487c-882b-2705ff70ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:06.304338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c6afc-ea89-487c-882b-2705ff70ae8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:06.315715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a724e-d13c-4b6e-a901-788cd485bf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:06.316570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a724e-d13c-4b6e-a901-788cd485bf55 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:06.322042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a724e-d13c-4b6e-a901-788cd485bf55 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:06.322042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a724e-d13c-4b6e-a901-788cd485bf55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:06.322042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a724e-d13c-4b6e-a901-788cd485bf55 None admin] RBAC: Adding query filter params (role.id=3455a742ed0a4787a4e60d7f104a586a, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=54b05cbc41b844d8932ba4c15d960289) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:06.322042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a724e-d13c-4b6e-a901-788cd485bf55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:06.322042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a724e-d13c-4b6e-a901-788cd485bf55 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:06.326723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 555/1103] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:42:06 2022] GET /identity/v3/role_assignments?user.id=54b05cbc41b844d8932ba4c15d960289&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=3455a742ed0a4787a4e60d7f104a586a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:42:06.374658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c6afc-ea89-487c-882b-2705ff70ae8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:06.376136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c6afc-ea89-487c-882b-2705ff70ae8c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:06.481830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 549/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:42:06.487036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a724e-d13c-4b6e-a901-788cd485bf55 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:06.591077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a724e-d13c-4b6e-a901-788cd485bf55 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:06.626864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433118a-782e-443a-bbc8-4a73175ea605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:06.627490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433118a-782e-443a-bbc8-4a73175ea605 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:06.627770 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433118a-782e-443a-bbc8-4a73175ea605 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:06.628042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433118a-782e-443a-bbc8-4a73175ea605 None admin] PATH_INFO: `/v3/projects/36339ca9f884448a9a08bc8ce77a9b13` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:06.629184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433118a-782e-443a-bbc8-4a73175ea605 None admin] RBAC: Authorizing `identity:get_project(project_id=36339ca9f884448a9a08bc8ce77a9b13)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:06.630678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433118a-782e-443a-bbc8-4a73175ea605 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:06.632310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 556/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:06 2022] GET /identity/v3/projects/36339ca9f884448a9a08bc8ce77a9b13 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:06.638531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c6afc-ea89-487c-882b-2705ff70ae8c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:06.659419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:06.741877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c6afc-ea89-487c-882b-2705ff70ae8c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:06.776459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e88bc2-9bfc-42db-bfff-bee9ad2fcc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:06.777060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e88bc2-9bfc-42db-bfff-bee9ad2fcc2d None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:06.777337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e88bc2-9bfc-42db-bfff-bee9ad2fcc2d None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:06.777606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e88bc2-9bfc-42db-bfff-bee9ad2fcc2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:06.778391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e88bc2-9bfc-42db-bfff-bee9ad2fcc2d None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=36339ca9f884448a9a08bc8ce77a9b13, user.id=6324fb13cedb48e9ae8d0f955270b25b) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:06.778697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e88bc2-9bfc-42db-bfff-bee9ad2fcc2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:06.780101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e88bc2-9bfc-42db-bfff-bee9ad2fcc2d None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:06.785914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 550/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:42:06 2022] GET /identity/v3/role_assignments?user.id=6324fb13cedb48e9ae8d0f955270b25b&scope.project.id=36339ca9f884448a9a08bc8ce77a9b13&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:08.807544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c05e88-b868-4b97-a653-3e77d93d26a2 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:08.808055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c05e88-b868-4b97-a653-3e77d93d26a2 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:08.808055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c05e88-b868-4b97-a653-3e77d93d26a2 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:08.809390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 557/1107] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:08.814678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467809-0215-44e3-ba3a-a839ff2dd092 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:08.815100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467809-0215-44e3-ba3a-a839ff2dd092 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:08.815227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467809-0215-44e3-ba3a-a839ff2dd092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:08.833740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-98467809-0215-44e3-ba3a-a839ff2dd092 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:08.834972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-98467809-0215-44e3-ba3a-a839ff2dd092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:08.901984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 551/1108] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:08.926255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c05e88-b868-4b97-a653-3e77d93d26a2 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:08.954339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:09.025867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c05e88-b868-4b97-a653-3e77d93d26a2 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:09.056954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cb19d-22bc-44b8-b324-f04962c39709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:09.057562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cb19d-22bc-44b8-b324-f04962c39709 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:09.057857 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cb19d-22bc-44b8-b324-f04962c39709 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:09.058123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cb19d-22bc-44b8-b324-f04962c39709 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:09.059195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cb19d-22bc-44b8-b324-f04962c39709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:09.060592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cb19d-22bc-44b8-b324-f04962c39709 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:09.062142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 558/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:42:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:09.068411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467809-0215-44e3-ba3a-a839ff2dd092 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:09.083797 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:09.164600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467809-0215-44e3-ba3a-a839ff2dd092 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:09.197422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c66be-3ffc-4737-8eda-efb5ff8350e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:09.198038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c66be-3ffc-4737-8eda-efb5ff8350e3 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:09.198318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c66be-3ffc-4737-8eda-efb5ff8350e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:09.198585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c66be-3ffc-4737-8eda-efb5ff8350e3 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:09.199213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c66be-3ffc-4737-8eda-efb5ff8350e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:09.200622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c66be-3ffc-4737-8eda-efb5ff8350e3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:09.277203 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 552/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:42:09 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:42:11.290220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bf42e-07f6-44cb-badb-94b82fd538a6 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:11.290743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bf42e-07f6-44cb-badb-94b82fd538a6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:11.290743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bf42e-07f6-44cb-badb-94b82fd538a6 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:11.292052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 559/1111] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:11.297185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186af8b-e4c8-4cf5-aa00-768e955b9152 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:11.297568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186af8b-e4c8-4cf5-aa00-768e955b9152 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:11.297732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186af8b-e4c8-4cf5-aa00-768e955b9152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:11.315353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5186af8b-e4c8-4cf5-aa00-768e955b9152 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:11.317087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5186af8b-e4c8-4cf5-aa00-768e955b9152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:11.378042 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:11.383076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 553/1112] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:11.406502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bf42e-07f6-44cb-badb-94b82fd538a6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:11.496771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:11.504574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bf42e-07f6-44cb-badb-94b82fd538a6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:11.536315 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071230c8-a303-4ace-b02d-196f19ef083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:11.536915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071230c8-a303-4ace-b02d-196f19ef083d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:11.537185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071230c8-a303-4ace-b02d-196f19ef083d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:11.537449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071230c8-a303-4ace-b02d-196f19ef083d None admin] PATH_INFO: `/v3/roles/d7d75f4c26eb44689ec71efb8b63e534` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:11.538932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071230c8-a303-4ace-b02d-196f19ef083d None admin] RBAC: Authorizing `identity:get_role(role_id=d7d75f4c26eb44689ec71efb8b63e534)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:11.540324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071230c8-a303-4ace-b02d-196f19ef083d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:11.541541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 560/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:11 2022] GET /identity/v3/roles/d7d75f4c26eb44689ec71efb8b63e534 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:11.547278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186af8b-e4c8-4cf5-aa00-768e955b9152 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:11.643085 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186af8b-e4c8-4cf5-aa00-768e955b9152 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:11.675829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40998518-a742-4619-8115-6b83bd095d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:11.676449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40998518-a742-4619-8115-6b83bd095d04 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:11.676724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40998518-a742-4619-8115-6b83bd095d04 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:11.676995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40998518-a742-4619-8115-6b83bd095d04 None admin] PATH_INFO: `/v3/users/de3187844e31493da734d844e538d233` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:11.689665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40998518-a742-4619-8115-6b83bd095d04 None admin] RBAC: Authorizing `identity:get_user(user_id=de3187844e31493da734d844e538d233)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:11.691724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40998518-a742-4619-8115-6b83bd095d04 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:11.693310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 554/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:11 2022] GET /identity/v3/users/de3187844e31493da734d844e538d233 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:11.699398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071230c8-a303-4ace-b02d-196f19ef083d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:11.793939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071230c8-a303-4ace-b02d-196f19ef083d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:11.827047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dabcc-bf96-4207-9ec7-0a7d447d0d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:11.827633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dabcc-bf96-4207-9ec7-0a7d447d0d4c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:11.827913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dabcc-bf96-4207-9ec7-0a7d447d0d4c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:11.828182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dabcc-bf96-4207-9ec7-0a7d447d0d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:11.856446 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:11.893087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dabcc-bf96-4207-9ec7-0a7d447d0d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:11.894575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dabcc-bf96-4207-9ec7-0a7d447d0d4c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:11.987479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 561/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:42:11.993818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40998518-a742-4619-8115-6b83bd095d04 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:12.051507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:12.090305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40998518-a742-4619-8115-6b83bd095d04 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:12.122953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea267a-3960-4212-8cc4-f388831fdde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:12.123565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea267a-3960-4212-8cc4-f388831fdde8 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:12.123840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea267a-3960-4212-8cc4-f388831fdde8 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:12.124104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea267a-3960-4212-8cc4-f388831fdde8 None admin] PATH_INFO: `/v3/projects/36339ca9f884448a9a08bc8ce77a9b13` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:12.125197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea267a-3960-4212-8cc4-f388831fdde8 None admin] RBAC: Authorizing `identity:get_project(project_id=36339ca9f884448a9a08bc8ce77a9b13)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:12.126620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea267a-3960-4212-8cc4-f388831fdde8 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:12.128266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 555/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:11 2022] GET /identity/v3/projects/36339ca9f884448a9a08bc8ce77a9b13 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:12.134389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dabcc-bf96-4207-9ec7-0a7d447d0d4c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:12.192515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:12.230239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dabcc-bf96-4207-9ec7-0a7d447d0d4c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:12.261445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b26352-50e8-4e82-854a-68c94faee95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:12.262057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b26352-50e8-4e82-854a-68c94faee95d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:12.262332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b26352-50e8-4e82-854a-68c94faee95d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:12.262600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b26352-50e8-4e82-854a-68c94faee95d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:12.263312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b26352-50e8-4e82-854a-68c94faee95d None admin] RBAC: Adding query filter params (role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=36339ca9f884448a9a08bc8ce77a9b13, user.id=de3187844e31493da734d844e538d233) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:12.263617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b26352-50e8-4e82-854a-68c94faee95d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:12.265035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b26352-50e8-4e82-854a-68c94faee95d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:12.270700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 562/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:42:12 2022] GET /identity/v3/role_assignments?user.id=de3187844e31493da734d844e538d233&scope.project.id=36339ca9f884448a9a08bc8ce77a9b13&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:14.226068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfef5de-da7b-44e3-9184-d5694edf6359 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:14.226701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfef5de-da7b-44e3-9184-d5694edf6359 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:14.226701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfef5de-da7b-44e3-9184-d5694edf6359 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:14.227952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 556/1118] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:14.232945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfcdeb-f8ba-4936-8644-79c9ae802ebf None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:14.233385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfcdeb-f8ba-4936-8644-79c9ae802ebf None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:14.233482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfcdeb-f8ba-4936-8644-79c9ae802ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:14.252357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c6dfcdeb-f8ba-4936-8644-79c9ae802ebf None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:14.253570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c6dfcdeb-f8ba-4936-8644-79c9ae802ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:14.317312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 563/1119] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:14.339697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfef5de-da7b-44e3-9184-d5694edf6359 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:14.439125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfef5de-da7b-44e3-9184-d5694edf6359 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:14.463858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:14.472598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff4d46-6456-475e-b8b8-62e782be5642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:14.480039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff4d46-6456-475e-b8b8-62e782be5642 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:14.480039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff4d46-6456-475e-b8b8-62e782be5642 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:14.480039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff4d46-6456-475e-b8b8-62e782be5642 None admin] PATH_INFO: `/v3/roles/d7d75f4c26eb44689ec71efb8b63e534` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:14.480039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff4d46-6456-475e-b8b8-62e782be5642 None admin] RBAC: Authorizing `identity:get_role(role_id=d7d75f4c26eb44689ec71efb8b63e534)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:14.480039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff4d46-6456-475e-b8b8-62e782be5642 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:14.480039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 557/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:14 2022] GET /identity/v3/roles/d7d75f4c26eb44689ec71efb8b63e534 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:14.483818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfcdeb-f8ba-4936-8644-79c9ae802ebf None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:14.538055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:14.579306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfcdeb-f8ba-4936-8644-79c9ae802ebf None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:14.612996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe154-65d1-4f54-be0a-508221f29aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:14.613618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe154-65d1-4f54-be0a-508221f29aaf None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:14.613929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe154-65d1-4f54-be0a-508221f29aaf None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:14.614200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe154-65d1-4f54-be0a-508221f29aaf None admin] PATH_INFO: `/v3/users/de3187844e31493da734d844e538d233` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:14.615298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fe154-65d1-4f54-be0a-508221f29aaf None admin] RBAC: Authorizing `identity:get_user(user_id=de3187844e31493da734d844e538d233)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:14.616710 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fe154-65d1-4f54-be0a-508221f29aaf None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:14.618300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 564/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:14 2022] GET /identity/v3/users/de3187844e31493da734d844e538d233 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:14.624077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff4d46-6456-475e-b8b8-62e782be5642 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:14.719040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff4d46-6456-475e-b8b8-62e782be5642 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:14.751522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f398302-150f-4e74-ab74-63210faa0b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:14.752068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f398302-150f-4e74-ab74-63210faa0b96 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:14.752339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f398302-150f-4e74-ab74-63210faa0b96 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:14.752602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f398302-150f-4e74-ab74-63210faa0b96 None admin] PATH_INFO: `/v3/projects/36339ca9f884448a9a08bc8ce77a9b13` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:14.753687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f398302-150f-4e74-ab74-63210faa0b96 None admin] RBAC: Authorizing `identity:get_project(project_id=36339ca9f884448a9a08bc8ce77a9b13)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:14.755115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f398302-150f-4e74-ab74-63210faa0b96 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:14.756692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 558/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:14 2022] GET /identity/v3/projects/36339ca9f884448a9a08bc8ce77a9b13 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:14.762468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe154-65d1-4f54-be0a-508221f29aaf None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:14.857418 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe154-65d1-4f54-be0a-508221f29aaf None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:14.879368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:14.889594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e65c3a-72d6-4fef-8118-06cb31b67115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:14.890334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e65c3a-72d6-4fef-8118-06cb31b67115 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:14.890602 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e65c3a-72d6-4fef-8118-06cb31b67115 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:14.890866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e65c3a-72d6-4fef-8118-06cb31b67115 None admin] PATH_INFO: `/v3/projects/36339ca9f884448a9a08bc8ce77a9b13/users/de3187844e31493da734d844e538d233/roles/d7d75f4c26eb44689ec71efb8b63e534` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:14.892927 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e65c3a-72d6-4fef-8118-06cb31b67115 None admin] RBAC: Authorizing `identity:create_grant(project_id=36339ca9f884448a9a08bc8ce77a9b13, user_id=de3187844e31493da734d844e538d233, role_id=d7d75f4c26eb44689ec71efb8b63e534)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:14.894957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e65c3a-72d6-4fef-8118-06cb31b67115 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:14.962698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 565/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:42:14 2022] PUT /identity/v3/projects/36339ca9f884448a9a08bc8ce77a9b13/users/de3187844e31493da734d844e538d233/roles/d7d75f4c26eb44689ec71efb8b63e534 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:42:16.930537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ff9bb-8445-4c97-abca-587f0057348e None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:16.931043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ff9bb-8445-4c97-abca-587f0057348e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:16.931043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ff9bb-8445-4c97-abca-587f0057348e None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:16.932343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 559/1124] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:16.937468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1f75-f4f2-4435-b00e-7663a3d9aab4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:16.937904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1f75-f4f2-4435-b00e-7663a3d9aab4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:16.938048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1f75-f4f2-4435-b00e-7663a3d9aab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:16.956224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-903b1f75-f4f2-4435-b00e-7663a3d9aab4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:16.957375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-903b1f75-f4f2-4435-b00e-7663a3d9aab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:17.020317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 566/1125] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:17.042842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ff9bb-8445-4c97-abca-587f0057348e None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:17.104571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:17.142503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ff9bb-8445-4c97-abca-587f0057348e None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:17.175341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544063cc-f060-4d8b-b7c5-7b9b8994bbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:17.175954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544063cc-f060-4d8b-b7c5-7b9b8994bbe9 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:17.176224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544063cc-f060-4d8b-b7c5-7b9b8994bbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:17.176478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544063cc-f060-4d8b-b7c5-7b9b8994bbe9 None admin] PATH_INFO: `/v3/roles/d7d75f4c26eb44689ec71efb8b63e534` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:17.177860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544063cc-f060-4d8b-b7c5-7b9b8994bbe9 None admin] RBAC: Authorizing `identity:get_role(role_id=d7d75f4c26eb44689ec71efb8b63e534)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:17.179193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544063cc-f060-4d8b-b7c5-7b9b8994bbe9 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:17.180489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 560/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:17 2022] GET /identity/v3/roles/d7d75f4c26eb44689ec71efb8b63e534 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:17.186503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1f75-f4f2-4435-b00e-7663a3d9aab4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:17.279881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1f75-f4f2-4435-b00e-7663a3d9aab4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:17.293583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:17.312980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098f9ae-13dc-40e5-bfb3-5caae12d5c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:17.313587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098f9ae-13dc-40e5-bfb3-5caae12d5c07 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:17.313887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098f9ae-13dc-40e5-bfb3-5caae12d5c07 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:17.314155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098f9ae-13dc-40e5-bfb3-5caae12d5c07 None admin] PATH_INFO: `/v3/users/de3187844e31493da734d844e538d233` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:17.315286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098f9ae-13dc-40e5-bfb3-5caae12d5c07 None admin] RBAC: Authorizing `identity:get_user(user_id=de3187844e31493da734d844e538d233)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:17.316698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098f9ae-13dc-40e5-bfb3-5caae12d5c07 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:17.318556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 567/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:17 2022] GET /identity/v3/users/de3187844e31493da734d844e538d233 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:17.324253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544063cc-f060-4d8b-b7c5-7b9b8994bbe9 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:17.419679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544063cc-f060-4d8b-b7c5-7b9b8994bbe9 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:17.448119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:17.453571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce0c02-a508-4d55-9673-378754fd3da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:17.454195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce0c02-a508-4d55-9673-378754fd3da4 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:17.461853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce0c02-a508-4d55-9673-378754fd3da4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:17.461853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce0c02-a508-4d55-9673-378754fd3da4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:17.518874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce0c02-a508-4d55-9673-378754fd3da4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:17.520306 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce0c02-a508-4d55-9673-378754fd3da4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:17.614440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 561/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:42:17.620241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098f9ae-13dc-40e5-bfb3-5caae12d5c07 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:17.717243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098f9ae-13dc-40e5-bfb3-5caae12d5c07 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:17.750337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990281b4-f184-401e-a662-bc73541e290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:17.750957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990281b4-f184-401e-a662-bc73541e290e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:17.751234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990281b4-f184-401e-a662-bc73541e290e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:17.751505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990281b4-f184-401e-a662-bc73541e290e None admin] PATH_INFO: `/v3/projects/36339ca9f884448a9a08bc8ce77a9b13` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:17.752613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990281b4-f184-401e-a662-bc73541e290e None admin] RBAC: Authorizing `identity:get_project(project_id=36339ca9f884448a9a08bc8ce77a9b13)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:17.754069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990281b4-f184-401e-a662-bc73541e290e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:17.755678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 568/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:17 2022] GET /identity/v3/projects/36339ca9f884448a9a08bc8ce77a9b13 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:17.761822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce0c02-a508-4d55-9673-378754fd3da4 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:17.790915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:17.857590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce0c02-a508-4d55-9673-378754fd3da4 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:17.890142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389df5b-090c-4f9f-a84a-d7f72716cf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:17.890727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389df5b-090c-4f9f-a84a-d7f72716cf26 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:17.891002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389df5b-090c-4f9f-a84a-d7f72716cf26 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:17.891301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389df5b-090c-4f9f-a84a-d7f72716cf26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:17.892014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389df5b-090c-4f9f-a84a-d7f72716cf26 None admin] RBAC: Adding query filter params (role.id=d7d75f4c26eb44689ec71efb8b63e534, scope.project.id=36339ca9f884448a9a08bc8ce77a9b13, user.id=de3187844e31493da734d844e538d233) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:17.892319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389df5b-090c-4f9f-a84a-d7f72716cf26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:17.894342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389df5b-090c-4f9f-a84a-d7f72716cf26 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:17.899879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 562/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:42:17 2022] GET /identity/v3/role_assignments?user.id=de3187844e31493da734d844e538d233&scope.project.id=36339ca9f884448a9a08bc8ce77a9b13&role.id=d7d75f4c26eb44689ec71efb8b63e534 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:19.899020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257cdb4-9d81-4d9a-941f-47931006eba3 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:19.899548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257cdb4-9d81-4d9a-941f-47931006eba3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:19.899548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257cdb4-9d81-4d9a-941f-47931006eba3 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:19.900812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 569/1131] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:19.906004 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ff5d7-e1f8-49ab-ae79-a6593b1be4db None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:19.906381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ff5d7-e1f8-49ab-ae79-a6593b1be4db None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:19.906543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ff5d7-e1f8-49ab-ae79-a6593b1be4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:19.924477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8a1ff5d7-e1f8-49ab-ae79-a6593b1be4db None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:19.925637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8a1ff5d7-e1f8-49ab-ae79-a6593b1be4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:19.988606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 563/1132] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:20.010745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257cdb4-9d81-4d9a-941f-47931006eba3 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:20.061097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:20.111971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257cdb4-9d81-4d9a-941f-47931006eba3 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:20.142997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f70c7b-a8e1-467b-b467-51cde29d22d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:20.144187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f70c7b-a8e1-467b-b467-51cde29d22d2 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:20.144494 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f70c7b-a8e1-467b-b467-51cde29d22d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:20.144762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f70c7b-a8e1-467b-b467-51cde29d22d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:20.145911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f70c7b-a8e1-467b-b467-51cde29d22d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:20.147339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f70c7b-a8e1-467b-b467-51cde29d22d2 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:20.148896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 570/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:42:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:20.154866 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ff5d7-e1f8-49ab-ae79-a6593b1be4db None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:20.178283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:20.249582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ff5d7-e1f8-49ab-ae79-a6593b1be4db None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:20.283527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff08791-6320-4f73-9cd6-003fa323b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:20.284163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff08791-6320-4f73-9cd6-003fa323b951 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:20.284435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff08791-6320-4f73-9cd6-003fa323b951 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:20.284700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff08791-6320-4f73-9cd6-003fa323b951 None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:20.285334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff08791-6320-4f73-9cd6-003fa323b951 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:20.286775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff08791-6320-4f73-9cd6-003fa323b951 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:20.369206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 564/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:42:20 2022] POST /identity/v3/projects => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:42:22.372428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefadbc-7b85-4495-b765-6d15997433db None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:22.372995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefadbc-7b85-4495-b765-6d15997433db None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:22.372995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefadbc-7b85-4495-b765-6d15997433db None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:22.374331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 571/1135] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:22.379966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c594-282d-4119-8c7f-b378fdfc70f8 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:22.380366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c594-282d-4119-8c7f-b378fdfc70f8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:22.380480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c594-282d-4119-8c7f-b378fdfc70f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:22.398873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-bf82c594-282d-4119-8c7f-b378fdfc70f8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:22.400047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-bf82c594-282d-4119-8c7f-b378fdfc70f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:22.464701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 565/1136] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:22.488448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefadbc-7b85-4495-b765-6d15997433db None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:22.587661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefadbc-7b85-4495-b765-6d15997433db None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:22.604330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:22.620204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d095a-4a5f-4f53-bca1-216898219901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:22.620817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d095a-4a5f-4f53-bca1-216898219901 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:22.621091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d095a-4a5f-4f53-bca1-216898219901 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:22.621358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d095a-4a5f-4f53-bca1-216898219901 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:22.622586 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d095a-4a5f-4f53-bca1-216898219901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:22.623864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d095a-4a5f-4f53-bca1-216898219901 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:22.625408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 572/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:42:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:22.631732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82c594-282d-4119-8c7f-b378fdfc70f8 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:22.639619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:22.726970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82c594-282d-4119-8c7f-b378fdfc70f8 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:22.759906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553aef9-2a11-485c-88b7-fc4fbb48ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:22.760496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553aef9-2a11-485c-88b7-fc4fbb48ad02 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:22.760767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553aef9-2a11-485c-88b7-fc4fbb48ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:22.761030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553aef9-2a11-485c-88b7-fc4fbb48ad02 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:22.761671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553aef9-2a11-485c-88b7-fc4fbb48ad02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:22.763109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553aef9-2a11-485c-88b7-fc4fbb48ad02 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:22.847523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 566/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:42:22 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:42:24.846838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8395-a23e-4262-9806-680b39a17404 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:24.847338 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8395-a23e-4262-9806-680b39a17404 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:24.847338 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8395-a23e-4262-9806-680b39a17404 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:24.848609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 573/1139] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:24.853768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13a6f4-34e1-4fe8-b782-f4efdd4f10c2 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:24.854213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13a6f4-34e1-4fe8-b782-f4efdd4f10c2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:24.854335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13a6f4-34e1-4fe8-b782-f4efdd4f10c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:24.871752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1a13a6f4-34e1-4fe8-b782-f4efdd4f10c2 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:24.872914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1a13a6f4-34e1-4fe8-b782-f4efdd4f10c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:24.926327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:24.935996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 567/1140] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:24.960307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a8395-a23e-4262-9806-680b39a17404 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:25.057686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a8395-a23e-4262-9806-680b39a17404 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:25.089937 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaa9b5-157a-413b-8f66-b51932235472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:25.090540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaa9b5-157a-413b-8f66-b51932235472 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:25.090812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaa9b5-157a-413b-8f66-b51932235472 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:25.091075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaa9b5-157a-413b-8f66-b51932235472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:25.100648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaa9b5-157a-413b-8f66-b51932235472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-dccaa9b5-157a-413b-8f66-b51932235472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:25.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:25.106312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:25.107529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 574/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:42:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:25.112615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13a6f4-34e1-4fe8-b782-f4efdd4f10c2 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:25.208017 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13a6f4-34e1-4fe8-b782-f4efdd4f10c2 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:25.240882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75008557-be30-444e-b6ef-5b2566400de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:25.241481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75008557-be30-444e-b6ef-5b2566400de9 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:25.241760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75008557-be30-444e-b6ef-5b2566400de9 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:25.242055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75008557-be30-444e-b6ef-5b2566400de9 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:25.242696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75008557-be30-444e-b6ef-5b2566400de9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:25.242997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75008557-be30-444e-b6ef-5b2566400de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:25.244351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75008557-be30-444e-b6ef-5b2566400de9 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:25.252152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 568/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:42:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:25.258456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaa9b5-157a-413b-8f66-b51932235472 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:25.304105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:25.355970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaa9b5-157a-413b-8f66-b51932235472 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:25.388120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a58a5e-5424-47a0-b31b-7465931f3845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:25.388731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a58a5e-5424-47a0-b31b-7465931f3845 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:25.389006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a58a5e-5424-47a0-b31b-7465931f3845 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:25.389274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a58a5e-5424-47a0-b31b-7465931f3845 None admin] PATH_INFO: `/v3/users/dff645d7a3134ee097095a2a64afb3d2` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:25.402117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a58a5e-5424-47a0-b31b-7465931f3845 None admin] RBAC: Authorizing `identity:get_user(user_id=dff645d7a3134ee097095a2a64afb3d2)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:25.403579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a58a5e-5424-47a0-b31b-7465931f3845 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:25.405139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 575/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:25 2022] GET /identity/v3/users/dff645d7a3134ee097095a2a64afb3d2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:25.411685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75008557-be30-444e-b6ef-5b2566400de9 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:25.465370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:25.508454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75008557-be30-444e-b6ef-5b2566400de9 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:25.540291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02e98b-e422-4504-aca0-12cbee280d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:25.540877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02e98b-e422-4504-aca0-12cbee280d98 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:25.541151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02e98b-e422-4504-aca0-12cbee280d98 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:25.541416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02e98b-e422-4504-aca0-12cbee280d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:25.605777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02e98b-e422-4504-aca0-12cbee280d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:25.607281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02e98b-e422-4504-aca0-12cbee280d98 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:25.657681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:25.701315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 569/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:42:25.707419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a58a5e-5424-47a0-b31b-7465931f3845 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:25.801673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a58a5e-5424-47a0-b31b-7465931f3845 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:25.803907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:25.834400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe224c6-c2b5-467f-8c2c-19cd071d9dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:25.835003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe224c6-c2b5-467f-8c2c-19cd071d9dce None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:25.835279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe224c6-c2b5-467f-8c2c-19cd071d9dce None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:25.835546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe224c6-c2b5-467f-8c2c-19cd071d9dce None admin] PATH_INFO: `/v3/projects/44ef821a970244939fc61a3846733899` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:25.836634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe224c6-c2b5-467f-8c2c-19cd071d9dce None admin] RBAC: Authorizing `identity:get_project(project_id=44ef821a970244939fc61a3846733899)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:25.838131 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe224c6-c2b5-467f-8c2c-19cd071d9dce None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:25.839768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 576/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:25 2022] GET /identity/v3/projects/44ef821a970244939fc61a3846733899 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:25.846454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02e98b-e422-4504-aca0-12cbee280d98 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:25.940130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02e98b-e422-4504-aca0-12cbee280d98 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:25.973106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07357106-1364-432f-95bf-b2b6bf7cfe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:25.973687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07357106-1364-432f-95bf-b2b6bf7cfe00 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:25.973987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07357106-1364-432f-95bf-b2b6bf7cfe00 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:25.974254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07357106-1364-432f-95bf-b2b6bf7cfe00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:25.974968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07357106-1364-432f-95bf-b2b6bf7cfe00 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=44ef821a970244939fc61a3846733899, user.id=dff645d7a3134ee097095a2a64afb3d2) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:25.975268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07357106-1364-432f-95bf-b2b6bf7cfe00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:25.976661 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07357106-1364-432f-95bf-b2b6bf7cfe00 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:25.981832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 570/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:42:25 2022] GET /identity/v3/role_assignments?user.id=dff645d7a3134ee097095a2a64afb3d2&scope.project.id=44ef821a970244939fc61a3846733899&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:27.924849 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb09a0b-4783-459e-8611-60bbe3f0d5c4 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:27.925364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb09a0b-4783-459e-8611-60bbe3f0d5c4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:27.925364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb09a0b-4783-459e-8611-60bbe3f0d5c4 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:27.926639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 577/1147] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:27.931657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28646d4b-2965-4081-8356-7f9c34502db5 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:27.932048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28646d4b-2965-4081-8356-7f9c34502db5 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:27.932198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28646d4b-2965-4081-8356-7f9c34502db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:27.950573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-28646d4b-2965-4081-8356-7f9c34502db5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:27.951773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-28646d4b-2965-4081-8356-7f9c34502db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:27.965083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:28.016723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 571/1148] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:28.038208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb09a0b-4783-459e-8611-60bbe3f0d5c4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:28.136017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb09a0b-4783-459e-8611-60bbe3f0d5c4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:28.172085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186fa37-6254-4f09-9371-7c7ef3bec2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:28.172696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186fa37-6254-4f09-9371-7c7ef3bec2ed None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:28.172973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186fa37-6254-4f09-9371-7c7ef3bec2ed None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:28.173241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186fa37-6254-4f09-9371-7c7ef3bec2ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:28.184424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186fa37-6254-4f09-9371-7c7ef3bec2ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-e186fa37-6254-4f09-9371-7c7ef3bec2ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:28.188891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:28.190795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:28.191925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 578/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:42:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:28.196748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28646d4b-2965-4081-8356-7f9c34502db5 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:28.290992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28646d4b-2965-4081-8356-7f9c34502db5 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:28.324193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e5537-fda2-4cb5-840a-14991662047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:28.324787 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e5537-fda2-4cb5-840a-14991662047f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:28.325060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e5537-fda2-4cb5-840a-14991662047f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:28.325327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e5537-fda2-4cb5-840a-14991662047f None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:28.326003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e5537-fda2-4cb5-840a-14991662047f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:28.326302 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e5537-fda2-4cb5-840a-14991662047f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:28.327649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e5537-fda2-4cb5-840a-14991662047f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:28.335348 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 572/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:42:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:28.341162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186fa37-6254-4f09-9371-7c7ef3bec2ed None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:28.372159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:28.437810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186fa37-6254-4f09-9371-7c7ef3bec2ed None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:28.469884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47a989-55e4-431a-b0fa-7025c58ff03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:28.470491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47a989-55e4-431a-b0fa-7025c58ff03a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:28.471283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47a989-55e4-431a-b0fa-7025c58ff03a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:28.471547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47a989-55e4-431a-b0fa-7025c58ff03a None admin] PATH_INFO: `/v3/users/dff645d7a3134ee097095a2a64afb3d2` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:28.472647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47a989-55e4-431a-b0fa-7025c58ff03a None admin] RBAC: Authorizing `identity:get_user(user_id=dff645d7a3134ee097095a2a64afb3d2)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:28.474085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47a989-55e4-431a-b0fa-7025c58ff03a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:28.475628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 579/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:28 2022] GET /identity/v3/users/dff645d7a3134ee097095a2a64afb3d2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:28.481422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e5537-fda2-4cb5-840a-14991662047f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:28.494429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:28.578884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e5537-fda2-4cb5-840a-14991662047f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:28.612348 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6e723-bf84-40e7-931e-dd73a72ea871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:28.612974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6e723-bf84-40e7-931e-dd73a72ea871 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:28.613252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6e723-bf84-40e7-931e-dd73a72ea871 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:28.613523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6e723-bf84-40e7-931e-dd73a72ea871 None admin] PATH_INFO: `/v3/projects/44ef821a970244939fc61a3846733899` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:28.614658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6e723-bf84-40e7-931e-dd73a72ea871 None admin] RBAC: Authorizing `identity:get_project(project_id=44ef821a970244939fc61a3846733899)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:28.616072 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6e723-bf84-40e7-931e-dd73a72ea871 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:28.617688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 573/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:28 2022] GET /identity/v3/projects/44ef821a970244939fc61a3846733899 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:28.623503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47a989-55e4-431a-b0fa-7025c58ff03a None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:28.716176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:28.718957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47a989-55e4-431a-b0fa-7025c58ff03a None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:28.753015 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332b180-0659-46eb-8488-8028a57cefb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:28.753646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332b180-0659-46eb-8488-8028a57cefb2 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:28.753945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332b180-0659-46eb-8488-8028a57cefb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:28.754211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332b180-0659-46eb-8488-8028a57cefb2 None admin] PATH_INFO: `/v3/projects/44ef821a970244939fc61a3846733899/users/dff645d7a3134ee097095a2a64afb3d2/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:28.756050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332b180-0659-46eb-8488-8028a57cefb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=44ef821a970244939fc61a3846733899, user_id=dff645d7a3134ee097095a2a64afb3d2, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:28.757455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332b180-0659-46eb-8488-8028a57cefb2 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:28.823450 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 580/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:42:28 2022] PUT /identity/v3/projects/44ef821a970244939fc61a3846733899/users/dff645d7a3134ee097095a2a64afb3d2/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:42:30.773207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d102a-538f-4fe9-afe6-d6849b8fe8fd None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:30.773738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d102a-538f-4fe9-afe6-d6849b8fe8fd None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:30.773738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d102a-538f-4fe9-afe6-d6849b8fe8fd None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:30.775114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 574/1154] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:30.780276 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c156dd-8a19-4fb4-a400-b146bcfbad57 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:30.780712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c156dd-8a19-4fb4-a400-b146bcfbad57 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:30.780806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c156dd-8a19-4fb4-a400-b146bcfbad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:30.799349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a2c156dd-8a19-4fb4-a400-b146bcfbad57 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:30.800513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a2c156dd-8a19-4fb4-a400-b146bcfbad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:30.863598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 581/1155] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:30.886030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d102a-538f-4fe9-afe6-d6849b8fe8fd None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:30.962819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:30.984541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d102a-538f-4fe9-afe6-d6849b8fe8fd None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:31.016544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d6fa7-5d97-45a6-b390-336efb88f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:31.017160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d6fa7-5d97-45a6-b390-336efb88f5dc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:31.017956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d6fa7-5d97-45a6-b390-336efb88f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:31.018224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d6fa7-5d97-45a6-b390-336efb88f5dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:31.028333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d6fa7-5d97-45a6-b390-336efb88f5dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-921d6fa7-5d97-45a6-b390-336efb88f5dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:31.032884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:31.034591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:31.035764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 575/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:31.040486 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c156dd-8a19-4fb4-a400-b146bcfbad57 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:31.124296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:31.133047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c156dd-8a19-4fb4-a400-b146bcfbad57 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:31.166163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcb7e2-e80d-4e0b-bef4-2ecb5cab478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:31.166746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcb7e2-e80d-4e0b-bef4-2ecb5cab478b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:31.167016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcb7e2-e80d-4e0b-bef4-2ecb5cab478b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:31.167279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcb7e2-e80d-4e0b-bef4-2ecb5cab478b None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:31.167925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcb7e2-e80d-4e0b-bef4-2ecb5cab478b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:31.168219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcb7e2-e80d-4e0b-bef4-2ecb5cab478b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:31.169553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcb7e2-e80d-4e0b-bef4-2ecb5cab478b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:31.177096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 582/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:42:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:31.182699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d6fa7-5d97-45a6-b390-336efb88f5dc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:31.279676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d6fa7-5d97-45a6-b390-336efb88f5dc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:31.312370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd73d4-a33e-4f0b-84e0-1394b1515d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:31.312984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd73d4-a33e-4f0b-84e0-1394b1515d69 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:31.313265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd73d4-a33e-4f0b-84e0-1394b1515d69 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:31.313527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd73d4-a33e-4f0b-84e0-1394b1515d69 None admin] PATH_INFO: `/v3/users/dff645d7a3134ee097095a2a64afb3d2` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:31.314733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd73d4-a33e-4f0b-84e0-1394b1515d69 None admin] RBAC: Authorizing `identity:get_user(user_id=dff645d7a3134ee097095a2a64afb3d2)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:31.316175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd73d4-a33e-4f0b-84e0-1394b1515d69 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:31.318052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 576/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:31 2022] GET /identity/v3/users/dff645d7a3134ee097095a2a64afb3d2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:31.323825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcb7e2-e80d-4e0b-bef4-2ecb5cab478b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:31.419982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcb7e2-e80d-4e0b-bef4-2ecb5cab478b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:31.452308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee1948-c19d-425d-a810-83ddbee3388e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:31.452890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee1948-c19d-425d-a810-83ddbee3388e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:31.453165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee1948-c19d-425d-a810-83ddbee3388e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:31.453433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee1948-c19d-425d-a810-83ddbee3388e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:31.473748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:31.518069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee1948-c19d-425d-a810-83ddbee3388e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:31.519505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee1948-c19d-425d-a810-83ddbee3388e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:31.612717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 583/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:42:31.618603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd73d4-a33e-4f0b-84e0-1394b1515d69 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:31.628092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:31.715363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd73d4-a33e-4f0b-84e0-1394b1515d69 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:31.747449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460be743-a766-4b92-8d97-a20eb35efe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:31.748070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460be743-a766-4b92-8d97-a20eb35efe06 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:31.748348 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460be743-a766-4b92-8d97-a20eb35efe06 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:31.748616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460be743-a766-4b92-8d97-a20eb35efe06 None admin] PATH_INFO: `/v3/projects/44ef821a970244939fc61a3846733899` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:31.749710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460be743-a766-4b92-8d97-a20eb35efe06 None admin] RBAC: Authorizing `identity:get_project(project_id=44ef821a970244939fc61a3846733899)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:31.751157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460be743-a766-4b92-8d97-a20eb35efe06 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:31.753383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 577/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:31 2022] GET /identity/v3/projects/44ef821a970244939fc61a3846733899 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:31.759475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee1948-c19d-425d-a810-83ddbee3388e None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:31.809740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:31.854464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee1948-c19d-425d-a810-83ddbee3388e None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:31.887059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdd6c8-f846-46b8-8ebe-dbf0a7c0a172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:31.887636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd6c8-f846-46b8-8ebe-dbf0a7c0a172 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:31.887909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd6c8-f846-46b8-8ebe-dbf0a7c0a172 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:31.888172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd6c8-f846-46b8-8ebe-dbf0a7c0a172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:31.888876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdd6c8-f846-46b8-8ebe-dbf0a7c0a172 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=44ef821a970244939fc61a3846733899, user.id=dff645d7a3134ee097095a2a64afb3d2) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:31.889175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdd6c8-f846-46b8-8ebe-dbf0a7c0a172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:31.890594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdd6c8-f846-46b8-8ebe-dbf0a7c0a172 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:31.896164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 584/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:42:31 2022] GET /identity/v3/role_assignments?user.id=dff645d7a3134ee097095a2a64afb3d2&scope.project.id=44ef821a970244939fc61a3846733899&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:33.880894 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2561eb-5400-4ad8-9f4b-65fa6addb315 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:33.881435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2561eb-5400-4ad8-9f4b-65fa6addb315 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:33.881435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2561eb-5400-4ad8-9f4b-65fa6addb315 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:33.882692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 578/1162] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:33.887803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c01eb0-fbfa-4b22-b96c-02e9d09fd2b6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:33.888216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c01eb0-fbfa-4b22-b96c-02e9d09fd2b6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:33.888313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c01eb0-fbfa-4b22-b96c-02e9d09fd2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:33.906448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-80c01eb0-fbfa-4b22-b96c-02e9d09fd2b6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:33.907603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-80c01eb0-fbfa-4b22-b96c-02e9d09fd2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:33.970549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 585/1163] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:33.992561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2561eb-5400-4ad8-9f4b-65fa6addb315 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:34.066115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:34.090310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2561eb-5400-4ad8-9f4b-65fa6addb315 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:34.122819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c17e8c-d70d-4451-b932-81a82dc931bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:34.123408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c17e8c-d70d-4451-b932-81a82dc931bc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:34.123684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c17e8c-d70d-4451-b932-81a82dc931bc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:34.123953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c17e8c-d70d-4451-b932-81a82dc931bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:34.192593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c17e8c-d70d-4451-b932-81a82dc931bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:34.193980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c17e8c-d70d-4451-b932-81a82dc931bc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:34.263299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:34.287999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 579/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:42:34.293270 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c01eb0-fbfa-4b22-b96c-02e9d09fd2b6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:34.337609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:34.390704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c01eb0-fbfa-4b22-b96c-02e9d09fd2b6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:34.423037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1440d0-6296-4b0f-af6c-258d7649a906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:34.424256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1440d0-6296-4b0f-af6c-258d7649a906 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:34.424526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1440d0-6296-4b0f-af6c-258d7649a906 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:34.424789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1440d0-6296-4b0f-af6c-258d7649a906 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:34.431102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1440d0-6296-4b0f-af6c-258d7649a906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:34.432511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1440d0-6296-4b0f-af6c-258d7649a906 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-0b1440d0-6296-4b0f-af6c-258d7649a906 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:34.444140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:34.445301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:34.446203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 586/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:42:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:34.451235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c17e8c-d70d-4451-b932-81a82dc931bc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:34.550530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c17e8c-d70d-4451-b932-81a82dc931bc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:34.582942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066edc4-2969-4cf6-9928-7d4cdf03304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:34.583541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066edc4-2969-4cf6-9928-7d4cdf03304f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:34.583819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066edc4-2969-4cf6-9928-7d4cdf03304f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:34.584090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066edc4-2969-4cf6-9928-7d4cdf03304f None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:34.584722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066edc4-2969-4cf6-9928-7d4cdf03304f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:34.585029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066edc4-2969-4cf6-9928-7d4cdf03304f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:34.586426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066edc4-2969-4cf6-9928-7d4cdf03304f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:34.593591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 580/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Aug 8 16:42:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:36.548166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f611fc-0960-4133-a166-0f602eebdf1f None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:36.548722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f611fc-0960-4133-a166-0f602eebdf1f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:36.548722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f611fc-0960-4133-a166-0f602eebdf1f None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:36.550060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 587/1167] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:36.555906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5210e2d-fa0e-4ebd-b62f-504a33131be0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:36.556318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5210e2d-fa0e-4ebd-b62f-504a33131be0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:36.556446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5210e2d-fa0e-4ebd-b62f-504a33131be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:36.574364 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a5210e2d-fa0e-4ebd-b62f-504a33131be0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:36.575530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a5210e2d-fa0e-4ebd-b62f-504a33131be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:36.604664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:36.640889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 581/1168] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:36.664445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f611fc-0960-4133-a166-0f602eebdf1f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:36.762544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f611fc-0960-4133-a166-0f602eebdf1f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:36.769120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:36.794137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbf01f-d9da-40cf-90e3-6891ff06c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:36.794732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbf01f-d9da-40cf-90e3-6891ff06c2cd None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:36.795003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbf01f-d9da-40cf-90e3-6891ff06c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:36.795271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbf01f-d9da-40cf-90e3-6891ff06c2cd None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:36.795915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbf01f-d9da-40cf-90e3-6891ff06c2cd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:36.797266 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbf01f-d9da-40cf-90e3-6891ff06c2cd None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:36.865697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 588/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Mon Aug 8 16:42:36 2022] POST /identity/v3/domains => generated 267 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:42:38.860266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdb332-6651-4f98-ae3b-8d4d85384512 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:38.860800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdb332-6651-4f98-ae3b-8d4d85384512 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:38.860800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdb332-6651-4f98-ae3b-8d4d85384512 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:38.862105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 582/1170] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:38.867609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ca831-6047-40ff-a835-6a26ab9c01b9 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:38.867993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ca831-6047-40ff-a835-6a26ab9c01b9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:38.868159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ca831-6047-40ff-a835-6a26ab9c01b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:38.885560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-619ca831-6047-40ff-a835-6a26ab9c01b9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:38.886795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-619ca831-6047-40ff-a835-6a26ab9c01b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:38.950321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 589/1171] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:38.973700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdb332-6651-4f98-ae3b-8d4d85384512 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:39.071830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdb332-6651-4f98-ae3b-8d4d85384512 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:39.104847 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f3b18-b10a-4727-a5de-a47e5227c466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:39.105452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f3b18-b10a-4727-a5de-a47e5227c466 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:39.105728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f3b18-b10a-4727-a5de-a47e5227c466 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:39.106024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f3b18-b10a-4727-a5de-a47e5227c466 None admin] PATH_INFO: `/v3/domains/93bf6de64c0f452f9dd4695eda3fb205` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:39.116050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f3b18-b10a-4727-a5de-a47e5227c466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=93bf6de64c0f452f9dd4695eda3fb205)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:39.117464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f3b18-b10a-4727-a5de-a47e5227c466 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:39.119041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 583/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Aug 8 16:42:38 2022] GET /identity/v3/domains/93bf6de64c0f452f9dd4695eda3fb205 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:39.124646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ca831-6047-40ff-a835-6a26ab9c01b9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:39.212810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:39.218305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ca831-6047-40ff-a835-6a26ab9c01b9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:39.251730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d0755-ff82-425b-a0c7-9578af694767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:39.252325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d0755-ff82-425b-a0c7-9578af694767 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:39.252600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d0755-ff82-425b-a0c7-9578af694767 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:39.252870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d0755-ff82-425b-a0c7-9578af694767 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:39.253493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d0755-ff82-425b-a0c7-9578af694767 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:39.254944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d0755-ff82-425b-a0c7-9578af694767 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:39.360794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 590/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:42:39 2022] POST /identity/v3/projects => generated 361 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:42:41.355947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adaef08-4263-48fe-9733-f060b2927802 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:41.356473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adaef08-4263-48fe-9733-f060b2927802 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:41.356473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adaef08-4263-48fe-9733-f060b2927802 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:41.357841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 584/1174] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:41.363082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593c84e-e843-4dd5-b3c2-c690b3ff7135 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:41.363508 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593c84e-e843-4dd5-b3c2-c690b3ff7135 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:41.363627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593c84e-e843-4dd5-b3c2-c690b3ff7135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:41.381397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1593c84e-e843-4dd5-b3c2-c690b3ff7135 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:41.382641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1593c84e-e843-4dd5-b3c2-c690b3ff7135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:41.446335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 591/1175] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:41.472723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adaef08-4263-48fe-9733-f060b2927802 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:41.499793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:41.569538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adaef08-4263-48fe-9733-f060b2927802 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:41.601385 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1954b0-7106-42de-a7cf-25bf12172843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:41.602031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1954b0-7106-42de-a7cf-25bf12172843 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:41.602305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1954b0-7106-42de-a7cf-25bf12172843 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:41.602569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1954b0-7106-42de-a7cf-25bf12172843 None admin] PATH_INFO: `/v3/domains/93bf6de64c0f452f9dd4695eda3fb205` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:41.603652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1954b0-7106-42de-a7cf-25bf12172843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=93bf6de64c0f452f9dd4695eda3fb205)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:41.605059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1954b0-7106-42de-a7cf-25bf12172843 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:41.606676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 585/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Aug 8 16:42:41 2022] GET /identity/v3/domains/93bf6de64c0f452f9dd4695eda3fb205 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:41.612358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593c84e-e843-4dd5-b3c2-c690b3ff7135 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:41.686353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:41.708267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593c84e-e843-4dd5-b3c2-c690b3ff7135 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:41.740934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2c6b0-895f-44a2-95a5-b4d261b1210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:41.741534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2c6b0-895f-44a2-95a5-b4d261b1210b None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:41.741862 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2c6b0-895f-44a2-95a5-b4d261b1210b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:41.742120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2c6b0-895f-44a2-95a5-b4d261b1210b None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:41.742731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2c6b0-895f-44a2-95a5-b4d261b1210b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:41.744151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2c6b0-895f-44a2-95a5-b4d261b1210b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:41.852626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 592/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:42:41 2022] POST /identity/v3/users => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:42:43.870285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854eaac-adf7-4998-abe1-a464a1c04365 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:43.870793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854eaac-adf7-4998-abe1-a464a1c04365 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:43.870793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854eaac-adf7-4998-abe1-a464a1c04365 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:43.872090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 586/1178] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:43.877686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05f8d9-8d50-4469-8641-489d94f50291 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:43.878115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05f8d9-8d50-4469-8641-489d94f50291 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:43.878254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05f8d9-8d50-4469-8641-489d94f50291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:43.896731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2e05f8d9-8d50-4469-8641-489d94f50291 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:43.897922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2e05f8d9-8d50-4469-8641-489d94f50291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:43.961187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 593/1179] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:43.983358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854eaac-adf7-4998-abe1-a464a1c04365 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:44.075494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:44.082517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854eaac-adf7-4998-abe1-a464a1c04365 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:44.114154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cc384-01fc-4658-9bd0-d4f39611a490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:44.114760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cc384-01fc-4658-9bd0-d4f39611a490 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:44.115035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cc384-01fc-4658-9bd0-d4f39611a490 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:44.115302 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cc384-01fc-4658-9bd0-d4f39611a490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:44.124881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cc384-01fc-4658-9bd0-d4f39611a490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-5f8cc384-01fc-4658-9bd0-d4f39611a490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:44.130123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:44.131135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:44.132142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 587/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:42:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:44.137284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05f8d9-8d50-4469-8641-489d94f50291 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:44.198249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:44.237728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05f8d9-8d50-4469-8641-489d94f50291 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:44.269951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8cbe2-0578-4277-a80c-305b802c9080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:44.270543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8cbe2-0578-4277-a80c-305b802c9080 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:44.270810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8cbe2-0578-4277-a80c-305b802c9080 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:44.271070 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8cbe2-0578-4277-a80c-305b802c9080 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:44.271717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8cbe2-0578-4277-a80c-305b802c9080 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:44.272009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8cbe2-0578-4277-a80c-305b802c9080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:44.273353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8cbe2-0578-4277-a80c-305b802c9080 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:44.280930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 594/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:42:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:44.287456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cc384-01fc-4658-9bd0-d4f39611a490 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:44.384728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cc384-01fc-4658-9bd0-d4f39611a490 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:44.417925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de68e00-a490-4d93-b5a6-91dde10f651c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:44.418548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de68e00-a490-4d93-b5a6-91dde10f651c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:44.418824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de68e00-a490-4d93-b5a6-91dde10f651c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:44.419093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de68e00-a490-4d93-b5a6-91dde10f651c None admin] PATH_INFO: `/v3/users/c2f16536cfbb49e48c49dc3f4187b5a6` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:44.431674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de68e00-a490-4d93-b5a6-91dde10f651c None admin] RBAC: Authorizing `identity:get_user(user_id=c2f16536cfbb49e48c49dc3f4187b5a6)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:44.433097 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de68e00-a490-4d93-b5a6-91dde10f651c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:44.434705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 588/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:44 2022] GET /identity/v3/users/c2f16536cfbb49e48c49dc3f4187b5a6 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:44.440620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8cbe2-0578-4277-a80c-305b802c9080 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:44.542826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8cbe2-0578-4277-a80c-305b802c9080 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:44.568760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:44.576849 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53052dc7-4644-4840-9787-c2ab68495252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:44.577441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53052dc7-4644-4840-9787-c2ab68495252 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:44.577712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53052dc7-4644-4840-9787-c2ab68495252 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:44.578012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53052dc7-4644-4840-9787-c2ab68495252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:44.642917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53052dc7-4644-4840-9787-c2ab68495252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:44.644277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53052dc7-4644-4840-9787-c2ab68495252 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:44.739451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 595/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:42:44.745615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de68e00-a490-4d93-b5a6-91dde10f651c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:44.761069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:44.842241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de68e00-a490-4d93-b5a6-91dde10f651c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:44.874480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111f0cf-d088-4371-8fe3-f6a963a514ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:44.875095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111f0cf-d088-4371-8fe3-f6a963a514ea None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:44.875367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111f0cf-d088-4371-8fe3-f6a963a514ea None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:44.876294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111f0cf-d088-4371-8fe3-f6a963a514ea None admin] PATH_INFO: `/v3/projects/dba52b14658f4124bfdad51985a56606` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:44.877431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111f0cf-d088-4371-8fe3-f6a963a514ea None admin] RBAC: Authorizing `identity:get_project(project_id=dba52b14658f4124bfdad51985a56606)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:44.878862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111f0cf-d088-4371-8fe3-f6a963a514ea None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:44.880468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 589/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:44 2022] GET /identity/v3/projects/dba52b14658f4124bfdad51985a56606 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:44.886604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53052dc7-4644-4840-9787-c2ab68495252 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:44.912031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:44.981286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53052dc7-4644-4840-9787-c2ab68495252 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:45.014603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95ad0c-84f5-4f6e-ac5b-f23f11e930b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:45.015189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95ad0c-84f5-4f6e-ac5b-f23f11e930b4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:45.015466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95ad0c-84f5-4f6e-ac5b-f23f11e930b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:45.015736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95ad0c-84f5-4f6e-ac5b-f23f11e930b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:45.016510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95ad0c-84f5-4f6e-ac5b-f23f11e930b4 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=dba52b14658f4124bfdad51985a56606, user.id=c2f16536cfbb49e48c49dc3f4187b5a6) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:45.016815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95ad0c-84f5-4f6e-ac5b-f23f11e930b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:45.018256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95ad0c-84f5-4f6e-ac5b-f23f11e930b4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:45.023423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 596/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:42:44 2022] GET /identity/v3/role_assignments?user.id=c2f16536cfbb49e48c49dc3f4187b5a6&scope.project.id=dba52b14658f4124bfdad51985a56606&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:46.972440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608463-d911-4525-a26a-8ac33adb6d77 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:46.973030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608463-d911-4525-a26a-8ac33adb6d77 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:46.973030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608463-d911-4525-a26a-8ac33adb6d77 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:46.974331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 590/1186] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:46.979607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3eea-209e-4f0b-8b5a-b1593879f4e3 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:46.979982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3eea-209e-4f0b-8b5a-b1593879f4e3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:46.980158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3eea-209e-4f0b-8b5a-b1593879f4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:46.998835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-382a3eea-209e-4f0b-8b5a-b1593879f4e3 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:47.000015 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-382a3eea-209e-4f0b-8b5a-b1593879f4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:47.063368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 597/1187] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:47.086298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a608463-d911-4525-a26a-8ac33adb6d77 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:47.166970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:47.185531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a608463-d911-4525-a26a-8ac33adb6d77 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:47.218366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7099a3-bfd2-4996-be56-dace2eb6e311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:47.218986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7099a3-bfd2-4996-be56-dace2eb6e311 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:47.219259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7099a3-bfd2-4996-be56-dace2eb6e311 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:47.219524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7099a3-bfd2-4996-be56-dace2eb6e311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:47.229967 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7099a3-bfd2-4996-be56-dace2eb6e311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-cd7099a3-bfd2-4996-be56-dace2eb6e311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:47.234464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:47.235610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:42:47.237114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 591/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:42:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:47.242260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a3eea-209e-4f0b-8b5a-b1593879f4e3 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:47.262408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:47.335919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a3eea-209e-4f0b-8b5a-b1593879f4e3 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:47.368484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ae843-2b62-455f-ab72-07f7349f2690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:47.369065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ae843-2b62-455f-ab72-07f7349f2690 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:47.369339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ae843-2b62-455f-ab72-07f7349f2690 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:47.369603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ae843-2b62-455f-ab72-07f7349f2690 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:47.370279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ae843-2b62-455f-ab72-07f7349f2690 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:47.370575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ae843-2b62-455f-ab72-07f7349f2690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:47.371924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ae843-2b62-455f-ab72-07f7349f2690 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:47.379587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 598/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:47.385271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7099a3-bfd2-4996-be56-dace2eb6e311 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:47.480793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7099a3-bfd2-4996-be56-dace2eb6e311 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:47.512701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cbe4-c40f-4686-ac7f-6abaa49a2d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:47.513301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cbe4-c40f-4686-ac7f-6abaa49a2d04 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:47.513571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cbe4-c40f-4686-ac7f-6abaa49a2d04 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:47.513860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cbe4-c40f-4686-ac7f-6abaa49a2d04 None admin] PATH_INFO: `/v3/users/c2f16536cfbb49e48c49dc3f4187b5a6` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:47.514950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04cbe4-c40f-4686-ac7f-6abaa49a2d04 None admin] RBAC: Authorizing `identity:get_user(user_id=c2f16536cfbb49e48c49dc3f4187b5a6)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:47.516347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04cbe4-c40f-4686-ac7f-6abaa49a2d04 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:47.518435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 592/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:47 2022] GET /identity/v3/users/c2f16536cfbb49e48c49dc3f4187b5a6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:47.523981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ae843-2b62-455f-ab72-07f7349f2690 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:47.605181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:47.622363 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ae843-2b62-455f-ab72-07f7349f2690 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:47.654867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c50ef-6540-440c-940e-6af9beb2eb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:47.655478 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c50ef-6540-440c-940e-6af9beb2eb32 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:47.655751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c50ef-6540-440c-940e-6af9beb2eb32 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:47.656018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c50ef-6540-440c-940e-6af9beb2eb32 None admin] PATH_INFO: `/v3/projects/dba52b14658f4124bfdad51985a56606` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:47.657091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c50ef-6540-440c-940e-6af9beb2eb32 None admin] RBAC: Authorizing `identity:get_project(project_id=dba52b14658f4124bfdad51985a56606)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:47.659113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c50ef-6540-440c-940e-6af9beb2eb32 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:47.660698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 599/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:47 2022] GET /identity/v3/projects/dba52b14658f4124bfdad51985a56606 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:47.666599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cbe4-c40f-4686-ac7f-6abaa49a2d04 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:47.678862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:47.763314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cbe4-c40f-4686-ac7f-6abaa49a2d04 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:47.795764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13f82d-abec-4ff0-9227-ea00d2c768d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:47.796403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f82d-abec-4ff0-9227-ea00d2c768d4 None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:47.796677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f82d-abec-4ff0-9227-ea00d2c768d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:47.796940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f82d-abec-4ff0-9227-ea00d2c768d4 None admin] PATH_INFO: `/v3/projects/dba52b14658f4124bfdad51985a56606/users/c2f16536cfbb49e48c49dc3f4187b5a6/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:47.798819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13f82d-abec-4ff0-9227-ea00d2c768d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=dba52b14658f4124bfdad51985a56606, user_id=c2f16536cfbb49e48c49dc3f4187b5a6, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:47.800228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13f82d-abec-4ff0-9227-ea00d2c768d4 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:47.879215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 593/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:42:47 2022] PUT /identity/v3/projects/dba52b14658f4124bfdad51985a56606/users/c2f16536cfbb49e48c49dc3f4187b5a6/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:42:49.836932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738d820-d432-401a-a3c4-33e7c60b1407 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:49.837473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738d820-d432-401a-a3c4-33e7c60b1407 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:49.837473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738d820-d432-401a-a3c4-33e7c60b1407 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:49.838876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 600/1193] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:42:49.843803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b20ca4-c02c-4d53-913f-887c356f6223 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:49.844211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b20ca4-c02c-4d53-913f-887c356f6223 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:49.844346 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b20ca4-c02c-4d53-913f-887c356f6223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:49.862470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d7b20ca4-c02c-4d53-913f-887c356f6223 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:42:49.863645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d7b20ca4-c02c-4d53-913f-887c356f6223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:42:49.923607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:49.927944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 594/1194] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:42:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:42:49.951749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738d820-d432-401a-a3c4-33e7c60b1407 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:50.049669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738d820-d432-401a-a3c4-33e7c60b1407 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:50.082614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed85a99-1512-41e3-8f4a-bd2a2c99d4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:50.083225 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed85a99-1512-41e3-8f4a-bd2a2c99d4e2 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:50.083501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed85a99-1512-41e3-8f4a-bd2a2c99d4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:50.083766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed85a99-1512-41e3-8f4a-bd2a2c99d4e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:50.093381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed85a99-1512-41e3-8f4a-bd2a2c99d4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-aed85a99-1512-41e3-8f4a-bd2a2c99d4e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:42:50.097949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:42:50.099012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:42:50.100035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 601/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:42:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:42:50.104984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b20ca4-c02c-4d53-913f-887c356f6223 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:50.199069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b20ca4-c02c-4d53-913f-887c356f6223 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:50.231441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd8e8c-d18b-4389-9fa9-ecde1533dfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:50.232029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd8e8c-d18b-4389-9fa9-ecde1533dfe6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:50.232301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd8e8c-d18b-4389-9fa9-ecde1533dfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:50.232565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd8e8c-d18b-4389-9fa9-ecde1533dfe6 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:50.233217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd8e8c-d18b-4389-9fa9-ecde1533dfe6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:50.233511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd8e8c-d18b-4389-9fa9-ecde1533dfe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:50.234886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd8e8c-d18b-4389-9fa9-ecde1533dfe6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:50.243258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 595/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:50.249085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed85a99-1512-41e3-8f4a-bd2a2c99d4e2 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:50.261720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:50.347350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed85a99-1512-41e3-8f4a-bd2a2c99d4e2 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:50.380778 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c62d2-7ee4-48d9-a7b6-858a3e0208e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:50.381388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c62d2-7ee4-48d9-a7b6-858a3e0208e7 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:50.381663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c62d2-7ee4-48d9-a7b6-858a3e0208e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:50.381960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c62d2-7ee4-48d9-a7b6-858a3e0208e7 None admin] PATH_INFO: `/v3/users/c2f16536cfbb49e48c49dc3f4187b5a6` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:50.383067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c62d2-7ee4-48d9-a7b6-858a3e0208e7 None admin] RBAC: Authorizing `identity:get_user(user_id=c2f16536cfbb49e48c49dc3f4187b5a6)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:50.384480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c62d2-7ee4-48d9-a7b6-858a3e0208e7 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:50.386058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 602/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Aug 8 16:42:50 2022] GET /identity/v3/users/c2f16536cfbb49e48c49dc3f4187b5a6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:50.391898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd8e8c-d18b-4389-9fa9-ecde1533dfe6 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:50.451867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:50.488705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd8e8c-d18b-4389-9fa9-ecde1533dfe6 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:50.521618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdeb2e-76ff-47ca-99f1-d9dc7d6de60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:50.522228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdeb2e-76ff-47ca-99f1-d9dc7d6de60b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:50.522501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdeb2e-76ff-47ca-99f1-d9dc7d6de60b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:50.522763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdeb2e-76ff-47ca-99f1-d9dc7d6de60b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:50.588587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdeb2e-76ff-47ca-99f1-d9dc7d6de60b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:50.590069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdeb2e-76ff-47ca-99f1-d9dc7d6de60b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:50.648792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:50.685950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 596/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:42:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 16:42:50.692165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c62d2-7ee4-48d9-a7b6-858a3e0208e7 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:50.768525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:42:50.789731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c62d2-7ee4-48d9-a7b6-858a3e0208e7 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:50.821521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268b442-bb2c-498d-8ff2-6ee1a9fc7f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:50.822162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268b442-bb2c-498d-8ff2-6ee1a9fc7f9a None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:50.822436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268b442-bb2c-498d-8ff2-6ee1a9fc7f9a None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:50.822701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268b442-bb2c-498d-8ff2-6ee1a9fc7f9a None admin] PATH_INFO: `/v3/projects/dba52b14658f4124bfdad51985a56606` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:50.823787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268b442-bb2c-498d-8ff2-6ee1a9fc7f9a None admin] RBAC: Authorizing `identity:get_project(project_id=dba52b14658f4124bfdad51985a56606)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:50.825196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268b442-bb2c-498d-8ff2-6ee1a9fc7f9a None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:50.826811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 603/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Aug 8 16:42:50 2022] GET /identity/v3/projects/dba52b14658f4124bfdad51985a56606 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:42:50.833352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdeb2e-76ff-47ca-99f1-d9dc7d6de60b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:42:50.927634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdeb2e-76ff-47ca-99f1-d9dc7d6de60b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:42:50.960398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f302370-89ed-4915-a3ae-b7bf3dac97bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:42:50.960982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f302370-89ed-4915-a3ae-b7bf3dac97bd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:42:50.961250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f302370-89ed-4915-a3ae-b7bf3dac97bd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:42:50.961510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f302370-89ed-4915-a3ae-b7bf3dac97bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:42:50.962251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f302370-89ed-4915-a3ae-b7bf3dac97bd None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=dba52b14658f4124bfdad51985a56606, user.id=c2f16536cfbb49e48c49dc3f4187b5a6) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:42:50.962549 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f302370-89ed-4915-a3ae-b7bf3dac97bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:42:50.963939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f302370-89ed-4915-a3ae-b7bf3dac97bd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:42:50.969405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 597/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:42:50 2022] GET /identity/v3/role_assignments?user.id=c2f16536cfbb49e48c49dc3f4187b5a6&scope.project.id=dba52b14658f4124bfdad51985a56606&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:43:00.817480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=27 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:43:00.966912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=87 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:03.221202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda437e-7f16-42a5-b784-7cb05304887f None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:03.221806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda437e-7f16-42a5-b784-7cb05304887f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:03.221806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda437e-7f16-42a5-b784-7cb05304887f None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:03.223123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 604/1201] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:03.228311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa56ec-f176-4701-b188-0d95d6b5880a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:03.228736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa56ec-f176-4701-b188-0d95d6b5880a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:03.228852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa56ec-f176-4701-b188-0d95d6b5880a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:03.247562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-24fa56ec-f176-4701-b188-0d95d6b5880a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:03.248783 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-24fa56ec-f176-4701-b188-0d95d6b5880a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:03.314881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 598/1202] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:03.338883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda437e-7f16-42a5-b784-7cb05304887f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:03.439267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda437e-7f16-42a5-b784-7cb05304887f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:03.471917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328ddfc-201d-4a7e-9d66-633271a50b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:03.472466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328ddfc-201d-4a7e-9d66-633271a50b2d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:03.472742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328ddfc-201d-4a7e-9d66-633271a50b2d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:03.473006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328ddfc-201d-4a7e-9d66-633271a50b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:03.482697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328ddfc-201d-4a7e-9d66-633271a50b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:03.485354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328ddfc-201d-4a7e-9d66-633271a50b2d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-c328ddfc-201d-4a7e-9d66-633271a50b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:03.498931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:03.500753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:03.502068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:03.502068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:03.502068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:03.502068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 605/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:44:03.506937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa56ec-f176-4701-b188-0d95d6b5880a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:03.604112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa56ec-f176-4701-b188-0d95d6b5880a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:03.632526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:03.638763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5d3e-4d0d-4ebb-8c8b-40d650defac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:03.638763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5d3e-4d0d-4ebb-8c8b-40d650defac3 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:03.639009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5d3e-4d0d-4ebb-8c8b-40d650defac3 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:03.639114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5d3e-4d0d-4ebb-8c8b-40d650defac3 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:03.639722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5d3e-4d0d-4ebb-8c8b-40d650defac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:03.640021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5d3e-4d0d-4ebb-8c8b-40d650defac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:03.641400 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5d3e-4d0d-4ebb-8c8b-40d650defac3 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:03.652294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 599/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:44:03.658664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328ddfc-201d-4a7e-9d66-633271a50b2d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:03.710997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:03.755534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328ddfc-201d-4a7e-9d66-633271a50b2d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:03.788414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0acac-a561-47e0-a7da-4d5af456a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:03.789039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0acac-a561-47e0-a7da-4d5af456a12f None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:03.789316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0acac-a561-47e0-a7da-4d5af456a12f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:03.789583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0acac-a561-47e0-a7da-4d5af456a12f None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:03.790240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0acac-a561-47e0-a7da-4d5af456a12f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:03.791660 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0acac-a561-47e0-a7da-4d5af456a12f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:03.899783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 606/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Aug 8 16:44:03 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:44:05.927992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338294a4-0c8b-498c-9cbf-33acf2e93710 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:05.928523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338294a4-0c8b-498c-9cbf-33acf2e93710 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:05.928523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338294a4-0c8b-498c-9cbf-33acf2e93710 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:05.929843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 600/1206] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:05.935040 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840033f9-4a6e-4c13-b553-5bc61a04df00 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:05.935731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840033f9-4a6e-4c13-b553-5bc61a04df00 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:05.935997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840033f9-4a6e-4c13-b553-5bc61a04df00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:05.954767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-840033f9-4a6e-4c13-b553-5bc61a04df00 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:05.955924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-840033f9-4a6e-4c13-b553-5bc61a04df00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:06.020456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 607/1207] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:06.043637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338294a4-0c8b-498c-9cbf-33acf2e93710 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:06.144058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338294a4-0c8b-498c-9cbf-33acf2e93710 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:06.176789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da4fbd-2a0f-453c-8656-440fb49f6c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:06.177408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da4fbd-2a0f-453c-8656-440fb49f6c93 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:06.177687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da4fbd-2a0f-453c-8656-440fb49f6c93 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:06.177988 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da4fbd-2a0f-453c-8656-440fb49f6c93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:06.189402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da4fbd-2a0f-453c-8656-440fb49f6c93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-09da4fbd-2a0f-453c-8656-440fb49f6c93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:44:06.194106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:06.195227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:06.196286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 601/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:44:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:06.201490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840033f9-4a6e-4c13-b553-5bc61a04df00 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:06.239922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:06.296006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840033f9-4a6e-4c13-b553-5bc61a04df00 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:06.330223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c2c03-b062-4943-a972-d3c1fcb35911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:06.330821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c2c03-b062-4943-a972-d3c1fcb35911 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:06.331100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c2c03-b062-4943-a972-d3c1fcb35911 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:06.331466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c2c03-b062-4943-a972-d3c1fcb35911 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:06.332125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c2c03-b062-4943-a972-d3c1fcb35911 None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:06.332426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c2c03-b062-4943-a972-d3c1fcb35911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:06.333815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c2c03-b062-4943-a972-d3c1fcb35911 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:06.341483 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 608/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:44:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:06.347466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da4fbd-2a0f-453c-8656-440fb49f6c93 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:06.397965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:06.447319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da4fbd-2a0f-453c-8656-440fb49f6c93 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:06.479755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a97d5-5f41-4d39-a096-66721ddad52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:06.480374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a97d5-5f41-4d39-a096-66721ddad52a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:06.480646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a97d5-5f41-4d39-a096-66721ddad52a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:06.480915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a97d5-5f41-4d39-a096-66721ddad52a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:06.492473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a97d5-5f41-4d39-a096-66721ddad52a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:06.493948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a97d5-5f41-4d39-a096-66721ddad52a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a22a97d5-5f41-4d39-a096-66721ddad52a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:06.508427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:06.509535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:06.510656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:06.510656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:06.510656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:06.510656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 602/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:06.515486 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c2c03-b062-4943-a972-d3c1fcb35911 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:06.611869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:06.614756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c2c03-b062-4943-a972-d3c1fcb35911 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:06.646357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f1813-a541-469b-9f08-2c53b2b3e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:06.647016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f1813-a541-469b-9f08-2c53b2b3e2bc None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:06.647224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f1813-a541-469b-9f08-2c53b2b3e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:06.647489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f1813-a541-469b-9f08-2c53b2b3e2bc None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:06.648119 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f1813-a541-469b-9f08-2c53b2b3e2bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:06.648414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f1813-a541-469b-9f08-2c53b2b3e2bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:06.649801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f1813-a541-469b-9f08-2c53b2b3e2bc None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:06.661544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 609/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:06.667251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a97d5-5f41-4d39-a096-66721ddad52a None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:06.768292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a97d5-5f41-4d39-a096-66721ddad52a None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:06.769875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:06.807043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e8ea7-5705-491c-8422-f4814ea530dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:06.807657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e8ea7-5705-491c-8422-f4814ea530dc None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:06.807928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e8ea7-5705-491c-8422-f4814ea530dc None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:06.808193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e8ea7-5705-491c-8422-f4814ea530dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:06.815868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e8ea7-5705-491c-8422-f4814ea530dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:06.817288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e8ea7-5705-491c-8422-f4814ea530dc None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b26e8ea7-5705-491c-8422-f4814ea530dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:44:06.828105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:06.829281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:06.831466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 603/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:06.835178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f1813-a541-469b-9f08-2c53b2b3e2bc None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:06.933499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f1813-a541-469b-9f08-2c53b2b3e2bc None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:06.966818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256d029-96f3-44d2-8622-821f239b2f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:06.967441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256d029-96f3-44d2-8622-821f239b2f9b None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:06.967678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256d029-96f3-44d2-8622-821f239b2f9b None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:06.968609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256d029-96f3-44d2-8622-821f239b2f9b None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:06.969335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256d029-96f3-44d2-8622-821f239b2f9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:06.969635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256d029-96f3-44d2-8622-821f239b2f9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:06.971086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256d029-96f3-44d2-8622-821f239b2f9b None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:06.987459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 610/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Aug 8 16:44:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:06.993310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e8ea7-5705-491c-8422-f4814ea530dc None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:07.090943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e8ea7-5705-491c-8422-f4814ea530dc None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:07.124172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf2094-938a-4f2a-9efb-8b64cd73cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:07.124792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf2094-938a-4f2a-9efb-8b64cd73cc0f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:07.125068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf2094-938a-4f2a-9efb-8b64cd73cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:07.125333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf2094-938a-4f2a-9efb-8b64cd73cc0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:07.155079 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:07.192704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf2094-938a-4f2a-9efb-8b64cd73cc0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:07.194201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf2094-938a-4f2a-9efb-8b64cd73cc0f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:07.290333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 604/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:44:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:44:07.296224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256d029-96f3-44d2-8622-821f239b2f9b None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:07.315401 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:07.393810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256d029-96f3-44d2-8622-821f239b2f9b None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:07.427050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14635266-9eb1-4dc0-828f-d49b80cab6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:07.433521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14635266-9eb1-4dc0-828f-d49b80cab6d0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:07.433521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14635266-9eb1-4dc0-828f-d49b80cab6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:07.433521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14635266-9eb1-4dc0-828f-d49b80cab6d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:07.438597 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14635266-9eb1-4dc0-828f-d49b80cab6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:07.440011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14635266-9eb1-4dc0-828f-d49b80cab6d0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-14635266-9eb1-4dc0-828f-d49b80cab6d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:07.453978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:07.455135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:07.456255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:07.456255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:07.456255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:07.456255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 611/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:07.461579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf2094-938a-4f2a-9efb-8b64cd73cc0f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:07.522923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:07.559905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf2094-938a-4f2a-9efb-8b64cd73cc0f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:07.594538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a4b18-4e99-40ad-b8db-c19895e77dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:07.595140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a4b18-4e99-40ad-b8db-c19895e77dc9 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:07.595411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a4b18-4e99-40ad-b8db-c19895e77dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:07.595671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a4b18-4e99-40ad-b8db-c19895e77dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:07.596445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a4b18-4e99-40ad-b8db-c19895e77dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:07.596744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a4b18-4e99-40ad-b8db-c19895e77dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:07.598129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a4b18-4e99-40ad-b8db-c19895e77dc9 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:07.609956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 605/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:07.615813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14635266-9eb1-4dc0-828f-d49b80cab6d0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:07.683632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:07.713408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14635266-9eb1-4dc0-828f-d49b80cab6d0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:07.746577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370ef33-0256-4261-9972-eccee8214d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:07.747194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370ef33-0256-4261-9972-eccee8214d53 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:07.747474 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370ef33-0256-4261-9972-eccee8214d53 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:07.747740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370ef33-0256-4261-9972-eccee8214d53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:07.754100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370ef33-0256-4261-9972-eccee8214d53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:07.755539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370ef33-0256-4261-9972-eccee8214d53 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-6370ef33-0256-4261-9972-eccee8214d53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:44:07.766714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:07.767757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:07.769008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:44:07.769008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:44:07.769008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:44:07.769008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:44:07.769008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:44:07.769008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:44:07.769008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:07.769008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:07.769008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 612/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:44:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:07.773665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a4b18-4e99-40ad-b8db-c19895e77dc9 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:07.874809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a4b18-4e99-40ad-b8db-c19895e77dc9 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:07.896503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:07.908293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813f93c-6f91-49af-a420-22546209682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:07.908900 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813f93c-6f91-49af-a420-22546209682a None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:07.909258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813f93c-6f91-49af-a420-22546209682a None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:07.909496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813f93c-6f91-49af-a420-22546209682a None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:07.910207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813f93c-6f91-49af-a420-22546209682a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:07.910510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813f93c-6f91-49af-a420-22546209682a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:07.911923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813f93c-6f91-49af-a420-22546209682a None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:07.923697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 606/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:44:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:07.929945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370ef33-0256-4261-9972-eccee8214d53 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:08.027519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370ef33-0256-4261-9972-eccee8214d53 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:08.054815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:08.060681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a239f-c6b3-41b2-b91f-c1b5cde17260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:08.061268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a239f-c6b3-41b2-b91f-c1b5cde17260 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:08.061540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a239f-c6b3-41b2-b91f-c1b5cde17260 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:08.062343 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a239f-c6b3-41b2-b91f-c1b5cde17260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:08.063051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a239f-c6b3-41b2-b91f-c1b5cde17260 None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=5853f9b60cf34380b218a5cda94f587d) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:08.063351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a239f-c6b3-41b2-b91f-c1b5cde17260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:08.064866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a239f-c6b3-41b2-b91f-c1b5cde17260 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:08.069987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 613/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:44:07 2022] GET /identity/v3/role_assignments?user.id=5853f9b60cf34380b218a5cda94f587d&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:10.038792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba41fcf-3e84-440b-b380-4f2e0ad15fec None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:10.039379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba41fcf-3e84-440b-b380-4f2e0ad15fec None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:10.039741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba41fcf-3e84-440b-b380-4f2e0ad15fec None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:10.042272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 607/1220] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:10.047778 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b89b58-7e2f-4ebb-a5d3-5c083fa08b2e None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:10.048179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b89b58-7e2f-4ebb-a5d3-5c083fa08b2e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:10.048321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b89b58-7e2f-4ebb-a5d3-5c083fa08b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:10.066662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-53b89b58-7e2f-4ebb-a5d3-5c083fa08b2e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:10.067824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-53b89b58-7e2f-4ebb-a5d3-5c083fa08b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:10.133422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 614/1221] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:10.155369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba41fcf-3e84-440b-b380-4f2e0ad15fec None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:10.258064 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba41fcf-3e84-440b-b380-4f2e0ad15fec None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:10.290394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ac457-6ec3-4be9-bf14-65099f0af085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:10.290970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ac457-6ec3-4be9-bf14-65099f0af085 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:10.291239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ac457-6ec3-4be9-bf14-65099f0af085 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:10.291501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ac457-6ec3-4be9-bf14-65099f0af085 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:10.301591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ac457-6ec3-4be9-bf14-65099f0af085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b02ac457-6ec3-4be9-bf14-65099f0af085 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:44:10.306199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:10.307268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:10.308174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 608/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:44:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:10.313583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b89b58-7e2f-4ebb-a5d3-5c083fa08b2e None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:10.407868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b89b58-7e2f-4ebb-a5d3-5c083fa08b2e None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:10.431111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:10.441493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecb465-0ca0-44ef-a25c-f4b35df1bd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:10.442117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecb465-0ca0-44ef-a25c-f4b35df1bd6c None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:10.442391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecb465-0ca0-44ef-a25c-f4b35df1bd6c None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:10.442654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecb465-0ca0-44ef-a25c-f4b35df1bd6c None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:10.443303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecb465-0ca0-44ef-a25c-f4b35df1bd6c None admin] RBAC: Adding query filter params (name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:10.443594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecb465-0ca0-44ef-a25c-f4b35df1bd6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:10.444935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecb465-0ca0-44ef-a25c-f4b35df1bd6c None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:10.452579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 615/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:44:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:10.458474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ac457-6ec3-4be9-bf14-65099f0af085 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:10.503367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:10.559719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ac457-6ec3-4be9-bf14-65099f0af085 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:10.593182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96e3e-d001-4378-81bc-7481afee2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:10.593830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96e3e-d001-4378-81bc-7481afee2063 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:10.594111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96e3e-d001-4378-81bc-7481afee2063 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:10.594375 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96e3e-d001-4378-81bc-7481afee2063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:10.604392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca96e3e-d001-4378-81bc-7481afee2063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:10.605893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca96e3e-d001-4378-81bc-7481afee2063 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-2ca96e3e-d001-4378-81bc-7481afee2063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:10.623751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:10.624915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:10.625835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:10.625835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:10.625835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:10.625835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 609/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:10.630707 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecb465-0ca0-44ef-a25c-f4b35df1bd6c None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:10.726069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecb465-0ca0-44ef-a25c-f4b35df1bd6c None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:10.757372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39990ff4-b5e4-4ae9-9f23-fa87e46d5e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:10.757996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39990ff4-b5e4-4ae9-9f23-fa87e46d5e82 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:10.758269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39990ff4-b5e4-4ae9-9f23-fa87e46d5e82 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:10.758540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39990ff4-b5e4-4ae9-9f23-fa87e46d5e82 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:10.759165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39990ff4-b5e4-4ae9-9f23-fa87e46d5e82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:10.759461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39990ff4-b5e4-4ae9-9f23-fa87e46d5e82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:10.760800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39990ff4-b5e4-4ae9-9f23-fa87e46d5e82 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:10.772877 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 616/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:10.778519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96e3e-d001-4378-81bc-7481afee2063 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:10.870158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:10.876687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96e3e-d001-4378-81bc-7481afee2063 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:10.909680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d6b89-b2ad-41ad-9f8c-6c18291ad321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:10.910354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d6b89-b2ad-41ad-9f8c-6c18291ad321 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:10.910632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d6b89-b2ad-41ad-9f8c-6c18291ad321 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:10.910902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d6b89-b2ad-41ad-9f8c-6c18291ad321 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:10.918862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d6b89-b2ad-41ad-9f8c-6c18291ad321 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:10.920333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d6b89-b2ad-41ad-9f8c-6c18291ad321 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a10d6b89-b2ad-41ad-9f8c-6c18291ad321 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:44:10.931041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:10.932145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:10.933166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 610/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:10.938360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39990ff4-b5e4-4ae9-9f23-fa87e46d5e82 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:10.984421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:11.036670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39990ff4-b5e4-4ae9-9f23-fa87e46d5e82 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:11.069543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50e0e0-317a-4494-b85f-d95b10bc4021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:11.070210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50e0e0-317a-4494-b85f-d95b10bc4021 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:11.070426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50e0e0-317a-4494-b85f-d95b10bc4021 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:11.070687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50e0e0-317a-4494-b85f-d95b10bc4021 None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:11.071398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50e0e0-317a-4494-b85f-d95b10bc4021 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:11.071693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50e0e0-317a-4494-b85f-d95b10bc4021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:11.073097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50e0e0-317a-4494-b85f-d95b10bc4021 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:11.089830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 617/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Aug 8 16:44:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:11.095535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d6b89-b2ad-41ad-9f8c-6c18291ad321 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:11.193278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d6b89-b2ad-41ad-9f8c-6c18291ad321 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:11.225700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cd05b-873a-4581-811f-e16d3cdf6b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:11.226420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cd05b-873a-4581-811f-e16d3cdf6b22 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:11.226696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cd05b-873a-4581-811f-e16d3cdf6b22 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:11.226958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cd05b-873a-4581-811f-e16d3cdf6b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:11.236993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cd05b-873a-4581-811f-e16d3cdf6b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:11.238451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cd05b-873a-4581-811f-e16d3cdf6b22 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-a66cd05b-873a-4581-811f-e16d3cdf6b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:11.252449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:11.253416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:11.254526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:11.254526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:11.254526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:11.254526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 611/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:11.259223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50e0e0-317a-4494-b85f-d95b10bc4021 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:11.361009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50e0e0-317a-4494-b85f-d95b10bc4021 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:11.363462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:11.394814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d6697-d3fa-4caa-b8c8-aa227c1d8dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:11.395403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d6697-d3fa-4caa-b8c8-aa227c1d8dfa None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:11.395678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d6697-d3fa-4caa-b8c8-aa227c1d8dfa None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:11.395943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d6697-d3fa-4caa-b8c8-aa227c1d8dfa None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:11.396569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d6697-d3fa-4caa-b8c8-aa227c1d8dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:11.396864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d6697-d3fa-4caa-b8c8-aa227c1d8dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:11.398252 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d6697-d3fa-4caa-b8c8-aa227c1d8dfa None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:11.409520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 618/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:11.415039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cd05b-873a-4581-811f-e16d3cdf6b22 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:11.424698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:11.512673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cd05b-873a-4581-811f-e16d3cdf6b22 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:11.544039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d080d-cc8b-41b7-9655-496f7dbd518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:11.544651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d080d-cc8b-41b7-9655-496f7dbd518f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:11.544921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d080d-cc8b-41b7-9655-496f7dbd518f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:11.545183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d080d-cc8b-41b7-9655-496f7dbd518f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:11.552100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d080d-cc8b-41b7-9655-496f7dbd518f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:11.553502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d080d-cc8b-41b7-9655-496f7dbd518f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-c96d080d-cc8b-41b7-9655-496f7dbd518f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:44:11.566467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:11.567841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:11.568771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:44:11.568771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:44:11.568771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:44:11.568771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:44:11.568771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:44:11.568771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:44:11.568771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:11.568771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:11.568771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 612/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:44:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:44:11.573409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d6697-d3fa-4caa-b8c8-aa227c1d8dfa None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:11.672245 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d6697-d3fa-4caa-b8c8-aa227c1d8dfa None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:11.704875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec5a57-ea05-4250-a12f-f0cb57f92174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:11.705469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec5a57-ea05-4250-a12f-f0cb57f92174 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:11.705740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec5a57-ea05-4250-a12f-f0cb57f92174 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:11.706034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec5a57-ea05-4250-a12f-f0cb57f92174 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:11.706716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec5a57-ea05-4250-a12f-f0cb57f92174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:11.707012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec5a57-ea05-4250-a12f-f0cb57f92174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:11.708590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec5a57-ea05-4250-a12f-f0cb57f92174 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:11.720245 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 619/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:44:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:11.726398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d080d-cc8b-41b7-9655-496f7dbd518f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:11.790978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:11.824809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d080d-cc8b-41b7-9655-496f7dbd518f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:11.856535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b2e27-006e-4e5e-8cfa-b2fc2fef2ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:11.857248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b2e27-006e-4e5e-8cfa-b2fc2fef2ebe None admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:11.857518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b2e27-006e-4e5e-8cfa-b2fc2fef2ebe None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:11.857924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b2e27-006e-4e5e-8cfa-b2fc2fef2ebe None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/5853f9b60cf34380b218a5cda94f587d/roles/68040d0d137e4250a000b019915a68e2` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:11.872813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b2e27-006e-4e5e-8cfa-b2fc2fef2ebe None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=5853f9b60cf34380b218a5cda94f587d, role_id=68040d0d137e4250a000b019915a68e2)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:11.874991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b2e27-006e-4e5e-8cfa-b2fc2fef2ebe None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:11.980733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 613/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:44:11 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/5853f9b60cf34380b218a5cda94f587d/roles/68040d0d137e4250a000b019915a68e2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:44:13.966881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943a61e-82b5-46fb-a72b-1d5eae155ff9 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:13.967410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943a61e-82b5-46fb-a72b-1d5eae155ff9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:13.967530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943a61e-82b5-46fb-a72b-1d5eae155ff9 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:13.968822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 620/1233] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:13.973948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0e9ec-92a2-45b6-9204-7aa966a75a71 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:13.974354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0e9ec-92a2-45b6-9204-7aa966a75a71 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:13.974513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0e9ec-92a2-45b6-9204-7aa966a75a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:13.992451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c6b0e9ec-92a2-45b6-9204-7aa966a75a71 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:13.993608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c6b0e9ec-92a2-45b6-9204-7aa966a75a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:14.058984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 614/1234] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 16:44:14.082515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943a61e-82b5-46fb-a72b-1d5eae155ff9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:14.108781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:14.181077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943a61e-82b5-46fb-a72b-1d5eae155ff9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:14.212677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a861203-6b86-459e-9d74-1a476d31ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:14.213287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a861203-6b86-459e-9d74-1a476d31ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:14.213580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a861203-6b86-459e-9d74-1a476d31ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:14.213846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a861203-6b86-459e-9d74-1a476d31ddf1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:14.223920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a861203-6b86-459e-9d74-1a476d31ddf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-3a861203-6b86-459e-9d74-1a476d31ddf1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:44:14.228399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:14.229496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:14.230561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 621/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Aug 8 16:44:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:14.236588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0e9ec-92a2-45b6-9204-7aa966a75a71 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:14.282172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:14.332218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0e9ec-92a2-45b6-9204-7aa966a75a71 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:14.364770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf40a6-c30c-491c-97c2-6be40df7e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:14.365365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf40a6-c30c-491c-97c2-6be40df7e976 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:14.365642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf40a6-c30c-491c-97c2-6be40df7e976 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:14.365939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf40a6-c30c-491c-97c2-6be40df7e976 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:14.366593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf40a6-c30c-491c-97c2-6be40df7e976 None admin] RBAC: Adding query filter params (name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:14.366888 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf40a6-c30c-491c-97c2-6be40df7e976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:14.368228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf40a6-c30c-491c-97c2-6be40df7e976 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:14.375969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 615/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Aug 8 16:44:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:14.381646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a861203-6b86-459e-9d74-1a476d31ddf1 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:14.464423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:14.480408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a861203-6b86-459e-9d74-1a476d31ddf1 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:14.512429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623b15-b9a0-4857-98c4-2bd40907c32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:14.513044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623b15-b9a0-4857-98c4-2bd40907c32e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:14.513319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623b15-b9a0-4857-98c4-2bd40907c32e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:14.513587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623b15-b9a0-4857-98c4-2bd40907c32e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:14.523843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e623b15-b9a0-4857-98c4-2bd40907c32e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:14.525390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e623b15-b9a0-4857-98c4-2bd40907c32e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-4e623b15-b9a0-4857-98c4-2bd40907c32e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:14.540237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:14.541335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:14.542268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:14.542268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:14.542268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:14.542268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 622/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:14.547427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf40a6-c30c-491c-97c2-6be40df7e976 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:14.646730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf40a6-c30c-491c-97c2-6be40df7e976 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:14.679148 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a9111-a84f-485f-94b3-19e76373435e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:14.679741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a9111-a84f-485f-94b3-19e76373435e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:14.680059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a9111-a84f-485f-94b3-19e76373435e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:14.680327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a9111-a84f-485f-94b3-19e76373435e None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:14.680992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a9111-a84f-485f-94b3-19e76373435e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:14.681540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a9111-a84f-485f-94b3-19e76373435e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:14.682659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a9111-a84f-485f-94b3-19e76373435e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:14.694479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 616/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:14.699963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623b15-b9a0-4857-98c4-2bd40907c32e None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:14.753359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:14.799399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623b15-b9a0-4857-98c4-2bd40907c32e None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:14.833109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4af219-62a5-44fb-b6d4-b49d9817f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:14.833729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4af219-62a5-44fb-b6d4-b49d9817f4bd None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:14.834037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4af219-62a5-44fb-b6d4-b49d9817f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:14.834308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4af219-62a5-44fb-b6d4-b49d9817f4bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:14.836938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:14.841280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4af219-62a5-44fb-b6d4-b49d9817f4bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:14.842747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4af219-62a5-44fb-b6d4-b49d9817f4bd None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-9f4af219-62a5-44fb-b6d4-b49d9817f4bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:44:14.853726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:14.854847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:14.855826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 623/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:14.860690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a9111-a84f-485f-94b3-19e76373435e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:14.962148 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a9111-a84f-485f-94b3-19e76373435e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:14.997098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6c19a-8a2b-4bb8-8477-01911ae7b572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:14.997736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6c19a-8a2b-4bb8-8477-01911ae7b572 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:14.997990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6c19a-8a2b-4bb8-8477-01911ae7b572 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:14.998256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6c19a-8a2b-4bb8-8477-01911ae7b572 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:14.999015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6c19a-8a2b-4bb8-8477-01911ae7b572 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:14.999314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6c19a-8a2b-4bb8-8477-01911ae7b572 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:15.000726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6c19a-8a2b-4bb8-8477-01911ae7b572 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:15.017977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 617/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Aug 8 16:44:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:15.025391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4af219-62a5-44fb-b6d4-b49d9817f4bd None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:15.121623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4af219-62a5-44fb-b6d4-b49d9817f4bd None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:15.153763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25a9d4-d09d-4421-9a9d-45b072e401cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:15.154377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25a9d4-d09d-4421-9a9d-45b072e401cf None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:15.154651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25a9d4-d09d-4421-9a9d-45b072e401cf None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:15.155500 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25a9d4-d09d-4421-9a9d-45b072e401cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:15.213841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:15.220744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25a9d4-d09d-4421-9a9d-45b072e401cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:15.222260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25a9d4-d09d-4421-9a9d-45b072e401cf None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:15.316240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 624/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:44:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:44:15.322064 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6c19a-8a2b-4bb8-8477-01911ae7b572 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:15.349558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:15.419372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6c19a-8a2b-4bb8-8477-01911ae7b572 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:15.453876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f6d4f-b447-44f2-9681-6a59d4b24a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:15.454496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f6d4f-b447-44f2-9681-6a59d4b24a22 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:15.454776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f6d4f-b447-44f2-9681-6a59d4b24a22 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:15.455045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f6d4f-b447-44f2-9681-6a59d4b24a22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:15.464906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f6d4f-b447-44f2-9681-6a59d4b24a22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:15.466389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f6d4f-b447-44f2-9681-6a59d4b24a22 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-ae0f6d4f-b447-44f2-9681-6a59d4b24a22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:15.481233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:15.482368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:15.483529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:15.483529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:15.483529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:15.483529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 618/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:44:15.488102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25a9d4-d09d-4421-9a9d-45b072e401cf None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:15.579775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:15.585722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25a9d4-d09d-4421-9a9d-45b072e401cf None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:15.618687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987e759-f317-40aa-9939-8b4e560b5077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:15.619283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987e759-f317-40aa-9939-8b4e560b5077 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:15.619559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987e759-f317-40aa-9939-8b4e560b5077 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:15.619896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987e759-f317-40aa-9939-8b4e560b5077 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:15.620530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987e759-f317-40aa-9939-8b4e560b5077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:15.620827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987e759-f317-40aa-9939-8b4e560b5077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:15.622239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987e759-f317-40aa-9939-8b4e560b5077 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:15.634182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 625/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:15.640525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f6d4f-b447-44f2-9681-6a59d4b24a22 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:15.718681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:15.739161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f6d4f-b447-44f2-9681-6a59d4b24a22 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:15.771851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5aa43-6deb-44ac-bfa9-8f5fa76f754b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:15.772463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5aa43-6deb-44ac-bfa9-8f5fa76f754b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:15.772739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5aa43-6deb-44ac-bfa9-8f5fa76f754b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:15.773008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5aa43-6deb-44ac-bfa9-8f5fa76f754b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:15.779554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5aa43-6deb-44ac-bfa9-8f5fa76f754b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:15.781459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5aa43-6deb-44ac-bfa9-8f5fa76f754b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-9cc5aa43-6deb-44ac-bfa9-8f5fa76f754b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:44:15.792897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:15.793968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:15.795243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:44:15.795243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:44:15.795243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:44:15.795243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:44:15.795243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:44:15.795243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:44:15.795243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:15.795243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:15.795243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 619/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:44:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:15.799893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987e759-f317-40aa-9939-8b4e560b5077 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:15.897769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987e759-f317-40aa-9939-8b4e560b5077 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:15.933083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8121b1-3a09-4345-8e52-b6563bfefb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:15.933677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8121b1-3a09-4345-8e52-b6563bfefb51 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:15.933987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8121b1-3a09-4345-8e52-b6563bfefb51 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:15.934284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8121b1-3a09-4345-8e52-b6563bfefb51 None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:15.935214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8121b1-3a09-4345-8e52-b6563bfefb51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:15.935509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8121b1-3a09-4345-8e52-b6563bfefb51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:15.937615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8121b1-3a09-4345-8e52-b6563bfefb51 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:15.949874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 626/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:44:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:15.956018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5aa43-6deb-44ac-bfa9-8f5fa76f754b None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:16.052937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5aa43-6deb-44ac-bfa9-8f5fa76f754b None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:16.086801 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fbdc9-0232-46e0-b01d-3115da62277b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:16.087433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fbdc9-0232-46e0-b01d-3115da62277b None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:16.087710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fbdc9-0232-46e0-b01d-3115da62277b None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:16.087980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fbdc9-0232-46e0-b01d-3115da62277b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:16.088712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fbdc9-0232-46e0-b01d-3115da62277b None admin] RBAC: Adding query filter params (role.id=68040d0d137e4250a000b019915a68e2, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=5853f9b60cf34380b218a5cda94f587d) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:16.089017 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fbdc9-0232-46e0-b01d-3115da62277b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:16.090469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fbdc9-0232-46e0-b01d-3115da62277b None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:16.096753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 620/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:44:15 2022] GET /identity/v3/role_assignments?user.id=5853f9b60cf34380b218a5cda94f587d&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=68040d0d137e4250a000b019915a68e2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:18.131503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d9033-df48-4f88-a005-bc67d239a7a5 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:18.132089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d9033-df48-4f88-a005-bc67d239a7a5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:18.132089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d9033-df48-4f88-a005-bc67d239a7a5 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:18.133369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 627/1247] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:18.138423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c20f0-63e4-4d50-9265-98e40764bd6e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:18.138854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c20f0-63e4-4d50-9265-98e40764bd6e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:18.139494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c20f0-63e4-4d50-9265-98e40764bd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:18.145762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:18.158245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5d5c20f0-63e4-4d50-9265-98e40764bd6e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:18.159470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5d5c20f0-63e4-4d50-9265-98e40764bd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:18.224075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 621/1248] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:18.246684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d9033-df48-4f88-a005-bc67d239a7a5 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:18.265327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:18.347134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d9033-df48-4f88-a005-bc67d239a7a5 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:18.379618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fc488-a4cb-4afb-9d00-1a3075e9f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:18.380228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fc488-a4cb-4afb-9d00-1a3075e9f064 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:18.380498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fc488-a4cb-4afb-9d00-1a3075e9f064 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:18.380758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fc488-a4cb-4afb-9d00-1a3075e9f064 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:18.394123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fc488-a4cb-4afb-9d00-1a3075e9f064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-e88fc488-a4cb-4afb-9d00-1a3075e9f064 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:44:18.395911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:18.397186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:18.398429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 628/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:44:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 16:44:18.403097 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c20f0-63e4-4d50-9265-98e40764bd6e None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:18.498530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c20f0-63e4-4d50-9265-98e40764bd6e None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:18.527551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:18.532941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6833f48-439c-4048-b25b-e620bb199e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:18.533534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6833f48-439c-4048-b25b-e620bb199e42 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:18.533824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6833f48-439c-4048-b25b-e620bb199e42 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:18.534094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6833f48-439c-4048-b25b-e620bb199e42 None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:18.534844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6833f48-439c-4048-b25b-e620bb199e42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:18.535150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6833f48-439c-4048-b25b-e620bb199e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:18.536497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6833f48-439c-4048-b25b-e620bb199e42 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:18.544290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 622/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:44:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:18.551983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fc488-a4cb-4afb-9d00-1a3075e9f064 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:18.628188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:18.650801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fc488-a4cb-4afb-9d00-1a3075e9f064 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:18.682868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07de2-e908-452b-ad3b-195f0f46dea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:18.683479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07de2-e908-452b-ad3b-195f0f46dea3 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:18.683754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07de2-e908-452b-ad3b-195f0f46dea3 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:18.684019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07de2-e908-452b-ad3b-195f0f46dea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:18.694403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07de2-e908-452b-ad3b-195f0f46dea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:18.705706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07de2-e908-452b-ad3b-195f0f46dea3 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-85d07de2-e908-452b-ad3b-195f0f46dea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:18.710124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:18.711342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:18.712564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:18.712564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:18.712564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:18.712564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 629/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:18.716976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6833f48-439c-4048-b25b-e620bb199e42 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:18.814834 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6833f48-439c-4048-b25b-e620bb199e42 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:18.846943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47001862-ee6c-42dd-8104-d055ac98dcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:18.854663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47001862-ee6c-42dd-8104-d055ac98dcd6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:18.854663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47001862-ee6c-42dd-8104-d055ac98dcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:18.854663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47001862-ee6c-42dd-8104-d055ac98dcd6 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:18.854663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47001862-ee6c-42dd-8104-d055ac98dcd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:18.854663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47001862-ee6c-42dd-8104-d055ac98dcd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:18.854663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47001862-ee6c-42dd-8104-d055ac98dcd6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:18.861662 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 623/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:18.867581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07de2-e908-452b-ad3b-195f0f46dea3 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:18.963161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07de2-e908-452b-ad3b-195f0f46dea3 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:18.988461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:18.995917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0add3f-64a8-4650-89c4-fc261d7cee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:18.996555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0add3f-64a8-4650-89c4-fc261d7cee55 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:18.996964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0add3f-64a8-4650-89c4-fc261d7cee55 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:18.997231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0add3f-64a8-4650-89c4-fc261d7cee55 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:19.004894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0add3f-64a8-4650-89c4-fc261d7cee55 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:19.006346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0add3f-64a8-4650-89c4-fc261d7cee55 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-cf0add3f-64a8-4650-89c4-fc261d7cee55 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:44:19.017051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:19.021096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:19.022394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 630/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:19.024088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47001862-ee6c-42dd-8104-d055ac98dcd6 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:19.078775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:19.121189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47001862-ee6c-42dd-8104-d055ac98dcd6 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:19.154239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca70e6-ee85-4975-b5b3-e9065156768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:19.154852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca70e6-ee85-4975-b5b3-e9065156768f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:19.155121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca70e6-ee85-4975-b5b3-e9065156768f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:19.155382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca70e6-ee85-4975-b5b3-e9065156768f None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:19.156084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca70e6-ee85-4975-b5b3-e9065156768f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:19.156377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca70e6-ee85-4975-b5b3-e9065156768f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:19.157818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca70e6-ee85-4975-b5b3-e9065156768f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:19.174420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 624/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Aug 8 16:44:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:19.180220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0add3f-64a8-4650-89c4-fc261d7cee55 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:19.277137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0add3f-64a8-4650-89c4-fc261d7cee55 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:19.310452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1eb2a-78a6-44c5-bcd7-853f053a11eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:19.311189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1eb2a-78a6-44c5-bcd7-853f053a11eb None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:19.311521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1eb2a-78a6-44c5-bcd7-853f053a11eb None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:19.311738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1eb2a-78a6-44c5-bcd7-853f053a11eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:19.365309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:19.377239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1eb2a-78a6-44c5-bcd7-853f053a11eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:19.378704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1eb2a-78a6-44c5-bcd7-853f053a11eb None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:19.473693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 631/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:44:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:44:19.479507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca70e6-ee85-4975-b5b3-e9065156768f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:19.577406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca70e6-ee85-4975-b5b3-e9065156768f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:19.595812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:19.617471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b09ffc-82d0-4be8-b819-062ff302fa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:19.618126 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b09ffc-82d0-4be8-b819-062ff302fa06 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:19.618401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b09ffc-82d0-4be8-b819-062ff302fa06 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:19.618667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b09ffc-82d0-4be8-b819-062ff302fa06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:19.628344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b09ffc-82d0-4be8-b819-062ff302fa06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:19.629759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b09ffc-82d0-4be8-b819-062ff302fa06 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-d3b09ffc-82d0-4be8-b819-062ff302fa06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:19.648680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:19.649720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:19.650701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:19.650701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:19.650701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:19.651078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 625/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:19.657577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1eb2a-78a6-44c5-bcd7-853f053a11eb None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:19.742861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:19.754436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1eb2a-78a6-44c5-bcd7-853f053a11eb None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:19.788127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb296c-5a7a-46d6-8fe0-dc6550317e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:19.788719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb296c-5a7a-46d6-8fe0-dc6550317e08 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:19.789034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb296c-5a7a-46d6-8fe0-dc6550317e08 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:19.789301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb296c-5a7a-46d6-8fe0-dc6550317e08 None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:19.789972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb296c-5a7a-46d6-8fe0-dc6550317e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:19.790272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb296c-5a7a-46d6-8fe0-dc6550317e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:19.791687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb296c-5a7a-46d6-8fe0-dc6550317e08 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:19.802562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 632/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:19.808183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b09ffc-82d0-4be8-b819-062ff302fa06 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:19.907003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b09ffc-82d0-4be8-b819-062ff302fa06 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:19.939632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56303dd4-35be-4a4e-b2c0-6af20c68005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:19.940252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56303dd4-35be-4a4e-b2c0-6af20c68005f None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:19.940529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56303dd4-35be-4a4e-b2c0-6af20c68005f None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:19.940796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56303dd4-35be-4a4e-b2c0-6af20c68005f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:19.947144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56303dd4-35be-4a4e-b2c0-6af20c68005f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:19.948562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56303dd4-35be-4a4e-b2c0-6af20c68005f None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-56303dd4-35be-4a4e-b2c0-6af20c68005f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:44:19.959826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:19.969559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:19.970712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:44:19.970712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:44:19.970712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:44:19.970712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:44:19.970712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:44:19.970712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:44:19.970712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:19.970712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:19.970712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 626/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:44:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:19.971294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb296c-5a7a-46d6-8fe0-dc6550317e08 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:20.065980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb296c-5a7a-46d6-8fe0-dc6550317e08 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:20.098913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb8205-cd02-4c33-99e7-e0320eed358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:20.099515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb8205-cd02-4c33-99e7-e0320eed358f None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:20.099791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb8205-cd02-4c33-99e7-e0320eed358f None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:20.100058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb8205-cd02-4c33-99e7-e0320eed358f None admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:20.100736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb8205-cd02-4c33-99e7-e0320eed358f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:20.101034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb8205-cd02-4c33-99e7-e0320eed358f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:20.102499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb8205-cd02-4c33-99e7-e0320eed358f None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:20.114183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 633/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:44:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:44:20.120343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56303dd4-35be-4a4e-b2c0-6af20c68005f None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:20.146991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:20.217928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56303dd4-35be-4a4e-b2c0-6af20c68005f None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:20.251592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f4735-c280-4051-a889-2d9b26037831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:20.252189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f4735-c280-4051-a889-2d9b26037831 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:20.252463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f4735-c280-4051-a889-2d9b26037831 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:20.252730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f4735-c280-4051-a889-2d9b26037831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:20.253489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f4735-c280-4051-a889-2d9b26037831 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=5853f9b60cf34380b218a5cda94f587d) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:20.253818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f4735-c280-4051-a889-2d9b26037831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:20.255238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f4735-c280-4051-a889-2d9b26037831 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:20.260471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 627/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:44:20 2022] GET /identity/v3/role_assignments?user.id=5853f9b60cf34380b218a5cda94f587d&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:22.241207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692523-4c49-4160-a445-7cf60c9b92fb None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:22.241758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692523-4c49-4160-a445-7cf60c9b92fb None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:22.241758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692523-4c49-4160-a445-7cf60c9b92fb None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:22.243139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 634/1261] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:22.248480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ff1d3-cf0c-4c84-a3d4-67d86113c0ba None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:22.249503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ff1d3-cf0c-4c84-a3d4-67d86113c0ba None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:22.249636 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ff1d3-cf0c-4c84-a3d4-67d86113c0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:22.267394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c39ff1d3-cf0c-4c84-a3d4-67d86113c0ba None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:22.268561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c39ff1d3-cf0c-4c84-a3d4-67d86113c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:22.333443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 628/1262] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:22.355960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692523-4c49-4160-a445-7cf60c9b92fb None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:22.367274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:22.454577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692523-4c49-4160-a445-7cf60c9b92fb None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:22.488761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9618503-47a1-4be8-91b5-5a78db879877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:22.489372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9618503-47a1-4be8-91b5-5a78db879877 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:22.489646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9618503-47a1-4be8-91b5-5a78db879877 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:22.489948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9618503-47a1-4be8-91b5-5a78db879877 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:22.499945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9618503-47a1-4be8-91b5-5a78db879877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-c9618503-47a1-4be8-91b5-5a78db879877 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:44:22.504722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:22.506166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:22.507181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 635/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:44:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:22.512382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ff1d3-cf0c-4c84-a3d4-67d86113c0ba None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:22.534210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:22.616617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ff1d3-cf0c-4c84-a3d4-67d86113c0ba None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:22.649970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eabab-df1e-48c7-aa45-80d7d56b17aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:22.650563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eabab-df1e-48c7-aa45-80d7d56b17aa None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:22.650835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eabab-df1e-48c7-aa45-80d7d56b17aa None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:22.651101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eabab-df1e-48c7-aa45-80d7d56b17aa None admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:22.651749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eabab-df1e-48c7-aa45-80d7d56b17aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:22.652044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eabab-df1e-48c7-aa45-80d7d56b17aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:22.653569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eabab-df1e-48c7-aa45-80d7d56b17aa None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:22.662092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 629/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:44:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:22.668445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9618503-47a1-4be8-91b5-5a78db879877 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:22.733180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:22.766390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9618503-47a1-4be8-91b5-5a78db879877 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:22.798828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c5ae8-1771-41fe-9238-7ef95b3a885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:22.799437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c5ae8-1771-41fe-9238-7ef95b3a885d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:22.799710 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c5ae8-1771-41fe-9238-7ef95b3a885d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:22.799970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c5ae8-1771-41fe-9238-7ef95b3a885d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:22.810325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c5ae8-1771-41fe-9238-7ef95b3a885d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:22.811793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c5ae8-1771-41fe-9238-7ef95b3a885d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-773c5ae8-1771-41fe-9238-7ef95b3a885d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:22.825237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:22.826415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:22.827994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:22.827994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:22.827994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:22.827994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 636/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:22.832715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eabab-df1e-48c7-aa45-80d7d56b17aa None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:22.918837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:22.937050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eabab-df1e-48c7-aa45-80d7d56b17aa None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:22.969445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257b02d-24d1-41eb-9f76-781eb5025f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:22.970066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257b02d-24d1-41eb-9f76-781eb5025f3e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:22.970345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257b02d-24d1-41eb-9f76-781eb5025f3e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:22.970614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257b02d-24d1-41eb-9f76-781eb5025f3e None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:22.971246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257b02d-24d1-41eb-9f76-781eb5025f3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:22.971544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257b02d-24d1-41eb-9f76-781eb5025f3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:22.972886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257b02d-24d1-41eb-9f76-781eb5025f3e None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:22.984040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 630/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:22.990902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c5ae8-1771-41fe-9238-7ef95b3a885d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:23.089543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c5ae8-1771-41fe-9238-7ef95b3a885d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:23.108268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:23.121650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cebc78-7696-445c-8d25-c7e0585d16b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:23.122333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cebc78-7696-445c-8d25-c7e0585d16b0 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:23.122604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cebc78-7696-445c-8d25-c7e0585d16b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:23.122872 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cebc78-7696-445c-8d25-c7e0585d16b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:23.130382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cebc78-7696-445c-8d25-c7e0585d16b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:23.131859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cebc78-7696-445c-8d25-c7e0585d16b0 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-c4cebc78-7696-445c-8d25-c7e0585d16b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:44:23.143008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:23.144182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:23.145202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 637/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:23.149981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257b02d-24d1-41eb-9f76-781eb5025f3e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:23.247456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257b02d-24d1-41eb-9f76-781eb5025f3e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:23.280858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef57b6-646f-4e68-b588-764cef62d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:23.281452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef57b6-646f-4e68-b588-764cef62d055 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:23.281715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef57b6-646f-4e68-b588-764cef62d055 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:23.282032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef57b6-646f-4e68-b588-764cef62d055 None admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:23.282752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef57b6-646f-4e68-b588-764cef62d055 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:23.283055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef57b6-646f-4e68-b588-764cef62d055 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:23.284472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef57b6-646f-4e68-b588-764cef62d055 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:23.292806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:23.301132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 631/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Aug 8 16:44:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:23.306917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cebc78-7696-445c-8d25-c7e0585d16b0 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:23.405108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cebc78-7696-445c-8d25-c7e0585d16b0 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:23.439448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb85bf-8008-4913-b66f-9d0040f01350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:23.440064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb85bf-8008-4913-b66f-9d0040f01350 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:23.440335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb85bf-8008-4913-b66f-9d0040f01350 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:23.440600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb85bf-8008-4913-b66f-9d0040f01350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:23.451266 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb85bf-8008-4913-b66f-9d0040f01350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:23.452699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb85bf-8008-4913-b66f-9d0040f01350 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-fabb85bf-8008-4913-b66f-9d0040f01350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:23.466136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:23.467397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:23.468695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:23.468695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:23.468695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:23.468695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 638/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:23.473062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef57b6-646f-4e68-b588-764cef62d055 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:23.572558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef57b6-646f-4e68-b588-764cef62d055 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:23.605643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3564b3e-ad48-4ca2-9997-7ba470ef9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:23.606278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3564b3e-ad48-4ca2-9997-7ba470ef9a81 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:23.606552 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3564b3e-ad48-4ca2-9997-7ba470ef9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:23.606815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3564b3e-ad48-4ca2-9997-7ba470ef9a81 None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:23.607439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3564b3e-ad48-4ca2-9997-7ba470ef9a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:23.607735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3564b3e-ad48-4ca2-9997-7ba470ef9a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:23.609690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3564b3e-ad48-4ca2-9997-7ba470ef9a81 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:23.620728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 632/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:23.627126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb85bf-8008-4913-b66f-9d0040f01350 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:23.658132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:23.723738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb85bf-8008-4913-b66f-9d0040f01350 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:23.757267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb35b7-9464-4e05-a334-824f394a857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:23.757906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb35b7-9464-4e05-a334-824f394a857d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:23.758183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb35b7-9464-4e05-a334-824f394a857d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:23.758449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb35b7-9464-4e05-a334-824f394a857d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:23.764728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb35b7-9464-4e05-a334-824f394a857d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:23.766189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb35b7-9464-4e05-a334-824f394a857d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-54cb35b7-9464-4e05-a334-824f394a857d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:44:23.780665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:23.781921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:23.783138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:44:23.783138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:44:23.783138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:44:23.783138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:44:23.783138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:44:23.783138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:44:23.783138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:23.783138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:23.783138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 639/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:44:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:23.788142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3564b3e-ad48-4ca2-9997-7ba470ef9a81 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:23.850445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:23.892737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3564b3e-ad48-4ca2-9997-7ba470ef9a81 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:23.925858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70777210-c6b3-4f8e-9b76-fd5c762224a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:23.926480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70777210-c6b3-4f8e-9b76-fd5c762224a0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:23.926751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70777210-c6b3-4f8e-9b76-fd5c762224a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:23.927014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70777210-c6b3-4f8e-9b76-fd5c762224a0 None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:23.927692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70777210-c6b3-4f8e-9b76-fd5c762224a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:23.927989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70777210-c6b3-4f8e-9b76-fd5c762224a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:23.929395 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70777210-c6b3-4f8e-9b76-fd5c762224a0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:23.942565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 633/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:44:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:23.948757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb35b7-9464-4e05-a334-824f394a857d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:24.037452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:24.045809 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb35b7-9464-4e05-a334-824f394a857d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:24.080259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1a495-9e23-48bd-805d-d3a25c6f82a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:24.080896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1a495-9e23-48bd-805d-d3a25c6f82a4 None admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:24.081172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1a495-9e23-48bd-805d-d3a25c6f82a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:24.081440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1a495-9e23-48bd-805d-d3a25c6f82a4 None admin] PATH_INFO: `/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/5853f9b60cf34380b218a5cda94f587d/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:24.083336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1a495-9e23-48bd-805d-d3a25c6f82a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1078a85a3648d08fb0fc64519bdfd3, user_id=5853f9b60cf34380b218a5cda94f587d, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:24.084744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1a495-9e23-48bd-805d-d3a25c6f82a4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:24.163213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 640/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Aug 8 16:44:23 2022] PUT /identity/v3/projects/3c1078a85a3648d08fb0fc64519bdfd3/users/5853f9b60cf34380b218a5cda94f587d/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:44:26.130805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae25cf-0971-43da-9d9d-03e5fc02ea7c None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:26.131365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae25cf-0971-43da-9d9d-03e5fc02ea7c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:26.131365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae25cf-0971-43da-9d9d-03e5fc02ea7c None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:26.132677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 634/1274] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:26.137890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2454da2-a998-4c08-927d-9eafe678e3c3 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:26.139672 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2454da2-a998-4c08-927d-9eafe678e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:26.139672 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2454da2-a998-4c08-927d-9eafe678e3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:26.156614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c2454da2-a998-4c08-927d-9eafe678e3c3 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:26.157866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c2454da2-a998-4c08-927d-9eafe678e3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:26.222865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 641/1275] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:26.246179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae25cf-0971-43da-9d9d-03e5fc02ea7c None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:26.344436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae25cf-0971-43da-9d9d-03e5fc02ea7c None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:26.361531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:26.377322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cff752-1151-4085-8bac-6b5556a8715e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:26.377980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cff752-1151-4085-8bac-6b5556a8715e None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:26.378250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cff752-1151-4085-8bac-6b5556a8715e None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:26.378512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cff752-1151-4085-8bac-6b5556a8715e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:26.388359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cff752-1151-4085-8bac-6b5556a8715e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-32cff752-1151-4085-8bac-6b5556a8715e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 16:44:26.393507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:26.394656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:26.395895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 635/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Aug 8 16:44:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:26.400635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2454da2-a998-4c08-927d-9eafe678e3c3 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:26.487858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:26.504882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2454da2-a998-4c08-927d-9eafe678e3c3 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:26.538137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0261ba-f591-4c85-8e93-a1c9717d44e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:26.538725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0261ba-f591-4c85-8e93-a1c9717d44e4 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:26.538998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0261ba-f591-4c85-8e93-a1c9717d44e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:26.539261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0261ba-f591-4c85-8e93-a1c9717d44e4 None admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:26.540018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0261ba-f591-4c85-8e93-a1c9717d44e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:26.540320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0261ba-f591-4c85-8e93-a1c9717d44e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:26.541658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0261ba-f591-4c85-8e93-a1c9717d44e4 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:26.549800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 642/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Aug 8 16:44:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:26.555521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cff752-1151-4085-8bac-6b5556a8715e None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:26.654423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cff752-1151-4085-8bac-6b5556a8715e None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:26.688450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded87ef-1fdf-4d25-a7c4-890a3864e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:26.689065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded87ef-1fdf-4d25-a7c4-890a3864e38c None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:26.689341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded87ef-1fdf-4d25-a7c4-890a3864e38c None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:26.689608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded87ef-1fdf-4d25-a7c4-890a3864e38c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:26.699491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded87ef-1fdf-4d25-a7c4-890a3864e38c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:26.700915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded87ef-1fdf-4d25-a7c4-890a3864e38c None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-7ded87ef-1fdf-4d25-a7c4-890a3864e38c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:26.715423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:26.716672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:26.718286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:26.718286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:26.718286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:26.718286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 636/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:26.723024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0261ba-f591-4c85-8e93-a1c9717d44e4 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:26.818487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0261ba-f591-4c85-8e93-a1c9717d44e4 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:26.852096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c107d-04d5-4972-9592-7804d45c2c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:26.852688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c107d-04d5-4972-9592-7804d45c2c0d None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:26.852967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c107d-04d5-4972-9592-7804d45c2c0d None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:26.853237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c107d-04d5-4972-9592-7804d45c2c0d None admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:26.853907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c107d-04d5-4972-9592-7804d45c2c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:26.854210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c107d-04d5-4972-9592-7804d45c2c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:26.855656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c107d-04d5-4972-9592-7804d45c2c0d None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:26.867018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 643/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:26.872762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded87ef-1fdf-4d25-a7c4-890a3864e38c None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:26.903983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:26.973435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded87ef-1fdf-4d25-a7c4-890a3864e38c None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:27.005840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779ada1-a30f-4d95-9d94-3408eabc6d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:27.006454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779ada1-a30f-4d95-9d94-3408eabc6d74 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:27.006727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779ada1-a30f-4d95-9d94-3408eabc6d74 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:27.006992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779ada1-a30f-4d95-9d94-3408eabc6d74 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:27.015067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779ada1-a30f-4d95-9d94-3408eabc6d74 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:27.016480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779ada1-a30f-4d95-9d94-3408eabc6d74 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-3779ada1-a30f-4d95-9d94-3408eabc6d74 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 16:44:27.027196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:27.028332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:27.029699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 637/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:27.034172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c107d-04d5-4972-9592-7804d45c2c0d None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:27.038042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:27.137109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c107d-04d5-4972-9592-7804d45c2c0d None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:27.169928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a4675-be58-49bf-8ec3-af8182d6c4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:27.171027 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a4675-be58-49bf-8ec3-af8182d6c4ff None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:27.171265 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a4675-be58-49bf-8ec3-af8182d6c4ff None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:27.171531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a4675-be58-49bf-8ec3-af8182d6c4ff None admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:27.172242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a4675-be58-49bf-8ec3-af8182d6c4ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:27.172543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a4675-be58-49bf-8ec3-af8182d6c4ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:27.173986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a4675-be58-49bf-8ec3-af8182d6c4ff None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:27.191984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 644/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Aug 8 16:44:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:27.197913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779ada1-a30f-4d95-9d94-3408eabc6d74 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:27.284823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:27.294256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779ada1-a30f-4d95-9d94-3408eabc6d74 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:27.327650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29827074-f7d8-4b58-b3ca-cc7afde93ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:27.328252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29827074-f7d8-4b58-b3ca-cc7afde93ed6 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:27.328586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29827074-f7d8-4b58-b3ca-cc7afde93ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:27.328856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29827074-f7d8-4b58-b3ca-cc7afde93ed6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:27.396295 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29827074-f7d8-4b58-b3ca-cc7afde93ed6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:27.397976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29827074-f7d8-4b58-b3ca-cc7afde93ed6 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:27.484374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:27.494672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 638/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Aug 8 16:44:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 16:44:27.501247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a4675-be58-49bf-8ec3-af8182d6c4ff None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:27.565173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:27.603744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a4675-be58-49bf-8ec3-af8182d6c4ff None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:27.636126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e1e48-d0d1-477a-812e-a1b5dee8f037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:27.636737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e1e48-d0d1-477a-812e-a1b5dee8f037 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:27.637016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e1e48-d0d1-477a-812e-a1b5dee8f037 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:27.637281 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e1e48-d0d1-477a-812e-a1b5dee8f037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:27.648098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e1e48-d0d1-477a-812e-a1b5dee8f037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:27.649594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e1e48-d0d1-477a-812e-a1b5dee8f037 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-111e1e48-d0d1-477a-812e-a1b5dee8f037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 16:44:27.663654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:27.664749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 16:44:27.665881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 16:44:27.665881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 16:44:27.665881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:27.665881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 645/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:44:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:27.671526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29827074-f7d8-4b58-b3ca-cc7afde93ed6 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:27.767533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29827074-f7d8-4b58-b3ca-cc7afde93ed6 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:27.802041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73ed12-8492-4320-ba45-585b323138cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:27.802638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73ed12-8492-4320-ba45-585b323138cd None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:27.802911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73ed12-8492-4320-ba45-585b323138cd None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:27.803178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73ed12-8492-4320-ba45-585b323138cd None admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:27.803806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73ed12-8492-4320-ba45-585b323138cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:27.804098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73ed12-8492-4320-ba45-585b323138cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:27.805438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73ed12-8492-4320-ba45-585b323138cd None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:27.817000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 639/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Aug 8 16:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:27.822681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e1e48-d0d1-477a-812e-a1b5dee8f037 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:27.921392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e1e48-d0d1-477a-812e-a1b5dee8f037 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:27.934493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:27.954648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265d266-0a2f-4e66-92f9-6a9398f2b234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:27.955267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265d266-0a2f-4e66-92f9-6a9398f2b234 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:27.955541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265d266-0a2f-4e66-92f9-6a9398f2b234 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:27.955806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265d266-0a2f-4e66-92f9-6a9398f2b234 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:27.962114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265d266-0a2f-4e66-92f9-6a9398f2b234 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:27.963560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265d266-0a2f-4e66-92f9-6a9398f2b234 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application [None req-7265d266-0a2f-4e66-92f9-6a9398f2b234 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 16:44:27.975497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:27.976487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:27.977399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 16:44:27.977399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 16:44:27.977399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 16:44:27.977399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 16:44:27.977399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 16:44:27.977399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 16:44:27.977399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 16:44:27.977399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: ERROR keystone.server.flask.application Aug 08 16:44:27.977399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 646/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Aug 8 16:44:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:27.982803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73ed12-8492-4320-ba45-585b323138cd None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:28.032850 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:28.080699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73ed12-8492-4320-ba45-585b323138cd None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:28.112933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c547b9-3124-4319-923c-c9e724097676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:28.113536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c547b9-3124-4319-923c-c9e724097676 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:28.113835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c547b9-3124-4319-923c-c9e724097676 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:28.114112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c547b9-3124-4319-923c-c9e724097676 None admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:28.114799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c547b9-3124-4319-923c-c9e724097676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:28.115101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c547b9-3124-4319-923c-c9e724097676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:28.116513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c547b9-3124-4319-923c-c9e724097676 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:28.129480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 640/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Aug 8 16:44:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:28.135715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265d266-0a2f-4e66-92f9-6a9398f2b234 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:28.233393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265d266-0a2f-4e66-92f9-6a9398f2b234 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:28.266879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f549e-b42b-4d4d-8c15-20c96de268c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:28.267474 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f549e-b42b-4d4d-8c15-20c96de268c1 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:28.267751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f549e-b42b-4d4d-8c15-20c96de268c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:28.268611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f549e-b42b-4d4d-8c15-20c96de268c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:28.268733 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f549e-b42b-4d4d-8c15-20c96de268c1 None admin] RBAC: Adding query filter params (role.id=dac864cdcb1a4cc283d63a7f33d20d17, scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3, user.id=5853f9b60cf34380b218a5cda94f587d) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:28.269013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f549e-b42b-4d4d-8c15-20c96de268c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:28.270457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f549e-b42b-4d4d-8c15-20c96de268c1 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:28.276002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 647/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Aug 8 16:44:28 2022] GET /identity/v3/role_assignments?user.id=5853f9b60cf34380b218a5cda94f587d&scope.project.id=3c1078a85a3648d08fb0fc64519bdfd3&role.id=dac864cdcb1a4cc283d63a7f33d20d17 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 16:44:30.279070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc03016-cf53-4ad5-9e7f-6fabba626d19 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:30.280227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc03016-cf53-4ad5-9e7f-6fabba626d19 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:30.280227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc03016-cf53-4ad5-9e7f-6fabba626d19 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:30.281502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 641/1288] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:30.286653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5beef-6000-4c73-a1ef-d634d0575ac1 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:30.287055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5beef-6000-4c73-a1ef-d634d0575ac1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:30.287234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5beef-6000-4c73-a1ef-d634d0575ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:30.304875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0ec5beef-6000-4c73-a1ef-d634d0575ac1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:30.306088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0ec5beef-6000-4c73-a1ef-d634d0575ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:30.335428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:30.372298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 648/1289] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:30.394695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc03016-cf53-4ad5-9e7f-6fabba626d19 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:30.494067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc03016-cf53-4ad5-9e7f-6fabba626d19 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:30.504716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:30.527446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3c125-9bc9-4283-815b-9a6aca5a8fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:30.528088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3c125-9bc9-4283-815b-9a6aca5a8fed None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:30.528368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3c125-9bc9-4283-815b-9a6aca5a8fed None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:30.528637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3c125-9bc9-4283-815b-9a6aca5a8fed None admin] PATH_INFO: `/v3/services/placement` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:30.529258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3c125-9bc9-4283-815b-9a6aca5a8fed None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:30.530622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3c125-9bc9-4283-815b-9a6aca5a8fed None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-0ea3c125-9bc9-4283-815b-9a6aca5a8fed None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:30.539045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:30.540326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:30.541408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 642/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:44:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:30.546400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5beef-6000-4c73-a1ef-d634d0575ac1 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:30.646383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5beef-6000-4c73-a1ef-d634d0575ac1 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:30.679460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef339e-aca5-4bd4-9f76-6d78624ea78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:30.680051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef339e-aca5-4bd4-9f76-6d78624ea78e None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:30.680361 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef339e-aca5-4bd4-9f76-6d78624ea78e None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:30.680627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef339e-aca5-4bd4-9f76-6d78624ea78e None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:30.681354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef339e-aca5-4bd4-9f76-6d78624ea78e None admin] RBAC: Adding query filter params (name=placement) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:30.681721 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef339e-aca5-4bd4-9f76-6d78624ea78e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:30.683120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef339e-aca5-4bd4-9f76-6d78624ea78e None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:30.690655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 649/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Aug 8 16:44:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:30.696582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3c125-9bc9-4283-815b-9a6aca5a8fed None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:30.793981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3c125-9bc9-4283-815b-9a6aca5a8fed None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:30.827268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524bb4c-9e61-445d-9855-bc8dd3d595b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:30.827862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524bb4c-9e61-445d-9855-bc8dd3d595b0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:30.828187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524bb4c-9e61-445d-9855-bc8dd3d595b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:30.828457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524bb4c-9e61-445d-9855-bc8dd3d595b0 None admin] PATH_INFO: `/v3/services` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:30.829077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524bb4c-9e61-445d-9855-bc8dd3d595b0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:30.829377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524bb4c-9e61-445d-9855-bc8dd3d595b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:30.830747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524bb4c-9e61-445d-9855-bc8dd3d595b0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:30.835167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 643/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Aug 8 16:44:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:32.852783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a09ba-b00f-43b9-b239-fb4d5ec7833a None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:32.853305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a09ba-b00f-43b9-b239-fb4d5ec7833a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:32.853305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a09ba-b00f-43b9-b239-fb4d5ec7833a None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:32.854776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 650/1293] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:32.860742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02340faa-a324-4b7f-b990-c558a2986ca0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:32.861185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02340faa-a324-4b7f-b990-c558a2986ca0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:32.861290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02340faa-a324-4b7f-b990-c558a2986ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:32.880296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-02340faa-a324-4b7f-b990-c558a2986ca0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:32.885013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-02340faa-a324-4b7f-b990-c558a2986ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:32.916456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:32.955949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 644/1294] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:32.980156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a09ba-b00f-43b9-b239-fb4d5ec7833a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:33.014542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:33.092223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a09ba-b00f-43b9-b239-fb4d5ec7833a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:33.128379 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652bcce-1ead-4fa1-ba37-ed10221af716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:33.128913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652bcce-1ead-4fa1-ba37-ed10221af716 None admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:33.129187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652bcce-1ead-4fa1-ba37-ed10221af716 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:33.129455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652bcce-1ead-4fa1-ba37-ed10221af716 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:33.130138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652bcce-1ead-4fa1-ba37-ed10221af716 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:33.131522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652bcce-1ead-4fa1-ba37-ed10221af716 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:33.204173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 651/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Aug 8 16:44:32 2022] POST /identity/v3/services => generated 246 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:44:35.206033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef966bc-0abe-40a3-b994-3b428d3b98d0 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:35.206594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef966bc-0abe-40a3-b994-3b428d3b98d0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:35.206594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef966bc-0abe-40a3-b994-3b428d3b98d0 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:35.207889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 645/1296] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:35.213028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af74980-0e68-4fc9-9384-a6f8a74b393b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:35.213433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af74980-0e68-4fc9-9384-a6f8a74b393b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:35.213600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af74980-0e68-4fc9-9384-a6f8a74b393b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:35.231248 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1af74980-0e68-4fc9-9384-a6f8a74b393b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:35.232422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1af74980-0e68-4fc9-9384-a6f8a74b393b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:35.301805 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 652/1297] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:35.324746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef966bc-0abe-40a3-b994-3b428d3b98d0 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:35.424227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef966bc-0abe-40a3-b994-3b428d3b98d0 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:35.458208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a4b03-2465-40f7-883b-9919cb844203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:35.458772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a4b03-2465-40f7-883b-9919cb844203 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:35.459050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a4b03-2465-40f7-883b-9919cb844203 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:35.459343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a4b03-2465-40f7-883b-9919cb844203 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:35.459920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a4b03-2465-40f7-883b-9919cb844203 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:35.461270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a4b03-2465-40f7-883b-9919cb844203 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-dc8a4b03-2465-40f7-883b-9919cb844203 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:35.469838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:35.470985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:35.471867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 646/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:44:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:35.476968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af74980-0e68-4fc9-9384-a6f8a74b393b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:35.495198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:35.571617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af74980-0e68-4fc9-9384-a6f8a74b393b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:35.603626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f1731-fc3c-45ed-a5e9-effabcc306e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:35.604237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f1731-fc3c-45ed-a5e9-effabcc306e6 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:35.604514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f1731-fc3c-45ed-a5e9-effabcc306e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:35.604780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f1731-fc3c-45ed-a5e9-effabcc306e6 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:35.605459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f1731-fc3c-45ed-a5e9-effabcc306e6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:35.605757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f1731-fc3c-45ed-a5e9-effabcc306e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:35.607654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f1731-fc3c-45ed-a5e9-effabcc306e6 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:35.612526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 653/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Aug 8 16:44:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:35.618383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a4b03-2465-40f7-883b-9919cb844203 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:35.649295 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:35.714766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a4b03-2465-40f7-883b-9919cb844203 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:35.747578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597303e8-3156-4ef5-ab0b-4e501d3394f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:35.748721 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597303e8-3156-4ef5-ab0b-4e501d3394f0 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:35.748993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597303e8-3156-4ef5-ab0b-4e501d3394f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:35.749253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597303e8-3156-4ef5-ab0b-4e501d3394f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:35.750001 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597303e8-3156-4ef5-ab0b-4e501d3394f0 None admin] RBAC: Adding query filter params (interface=public, service_id=f179b7866f3e43e296c06f06054edd9d, region_id=RegionOne) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:35.750522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597303e8-3156-4ef5-ab0b-4e501d3394f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:35.752675 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597303e8-3156-4ef5-ab0b-4e501d3394f0 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:35.757537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 647/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Aug 8 16:44:35 2022] GET /identity/v3/endpoints?service_id=f179b7866f3e43e296c06f06054edd9d&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:35.763468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f1731-fc3c-45ed-a5e9-effabcc306e6 None admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:35.842812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:35.860143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f1731-fc3c-45ed-a5e9-effabcc306e6 None admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:35.892320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec065c1-f1e8-4e0c-ac68-89591f6fa671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:35.892914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec065c1-f1e8-4e0c-ac68-89591f6fa671 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:35.893193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec065c1-f1e8-4e0c-ac68-89591f6fa671 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:35.893461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec065c1-f1e8-4e0c-ac68-89591f6fa671 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:35.894079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec065c1-f1e8-4e0c-ac68-89591f6fa671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:35.895424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec065c1-f1e8-4e0c-ac68-89591f6fa671 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:35.902055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 654/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Aug 8 16:44:35 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:44:37.884698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2188e-6a4e-4a95-bf34-7027e456fb19 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:37.885212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2188e-6a4e-4a95-bf34-7027e456fb19 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:37.885212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2188e-6a4e-4a95-bf34-7027e456fb19 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:37.886528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 648/1302] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:44:37.892170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17940ece-b0ed-4882-b64e-03ee752c125e None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:37.892640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17940ece-b0ed-4882-b64e-03ee752c125e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:37.892744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17940ece-b0ed-4882-b64e-03ee752c125e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:37.913743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-17940ece-b0ed-4882-b64e-03ee752c125e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:44:37.915107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-17940ece-b0ed-4882-b64e-03ee752c125e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:44:37.980946 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 655/1303] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:44:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 16:44:38.003266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2188e-6a4e-4a95-bf34-7027e456fb19 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:38.095495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:38.101806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2188e-6a4e-4a95-bf34-7027e456fb19 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:38.133694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6bd7-8486-455c-adad-24eef0d56953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:38.134345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6bd7-8486-455c-adad-24eef0d56953 None admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:38.134619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6bd7-8486-455c-adad-24eef0d56953 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:38.134932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6bd7-8486-455c-adad-24eef0d56953 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:38.135455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6bd7-8486-455c-adad-24eef0d56953 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:38.136801 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6bd7-8486-455c-adad-24eef0d56953 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-b1fe6bd7-8486-455c-adad-24eef0d56953 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 16:44:38.145774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 16:44:38.146918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 16:44:38.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 649/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Aug 8 16:44:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 16:44:38.152803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17940ece-b0ed-4882-b64e-03ee752c125e None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:38.229166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:38.252649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17940ece-b0ed-4882-b64e-03ee752c125e None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:38.290238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66102226-9198-466d-9c5a-570984b460d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:38.290840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66102226-9198-466d-9c5a-570984b460d5 None admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:38.291115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66102226-9198-466d-9c5a-570984b460d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:38.291382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66102226-9198-466d-9c5a-570984b460d5 None admin] PATH_INFO: `/v3/services` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:38.292003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66102226-9198-466d-9c5a-570984b460d5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:44:38.292306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66102226-9198-466d-9c5a-570984b460d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:38.293687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66102226-9198-466d-9c5a-570984b460d5 None admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:38.298665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 656/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Aug 8 16:44:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:44:38.304499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6bd7-8486-455c-adad-24eef0d56953 None admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:44:38.401273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6bd7-8486-455c-adad-24eef0d56953 None admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:44:38.433577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2214290-11c3-4843-8d67-cb2312ded613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:44:38.434187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2214290-11c3-4843-8d67-cb2312ded613 None admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:44:38.434462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2214290-11c3-4843-8d67-cb2312ded613 None admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:44:38.434730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2214290-11c3-4843-8d67-cb2312ded613 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:44:38.436030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2214290-11c3-4843-8d67-cb2312ded613 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:44:38.437372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2214290-11c3-4843-8d67-cb2312ded613 None admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:44:38.510572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 650/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:44:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 16:44:48.295675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=30 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:44:48.506179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:30.278838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c8650-2e0a-4ca2-9f8b-479a9c225af5 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:30.279954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c8650-2e0a-4ca2-9f8b-479a9c225af5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:30.280465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c8650-2e0a-4ca2-9f8b-479a9c225af5 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:30.282413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 657/1307] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:30.287330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5f5d-11e8-4765-8a83-7c34552d7f1e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:30.287789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5f5d-11e8-4765-8a83-7c34552d7f1e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:30.287871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5f5d-11e8-4765-8a83-7c34552d7f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:30.387499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-de3b5f5d-11e8-4765-8a83-7c34552d7f1e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:30.388727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-de3b5f5d-11e8-4765-8a83-7c34552d7f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:30.521970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 651/1308] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:30.557295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248fd8d-f21b-4bc9-94d5-ae0b24099d1a None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:30.557605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248fd8d-f21b-4bc9-94d5-ae0b24099d1a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:30.557859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248fd8d-f21b-4bc9-94d5-ae0b24099d1a None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:30.559307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 658/1309] 10.222.0.57 () {54 vars in 965 bytes} [Mon Aug 8 16:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:30.566474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da705-ea9c-48d3-8e57-1243971d1274 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:30.566758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da705-ea9c-48d3-8e57-1243971d1274 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:30.567028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da705-ea9c-48d3-8e57-1243971d1274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:30.649036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-470da705-ea9c-48d3-8e57-1243971d1274 None None] MFA Rules not processed for user `f50ee57e2418417cbe68915ca3f00f76`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:30.650273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-470da705-ea9c-48d3-8e57-1243971d1274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:30.673247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:30.731314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 652/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Aug 8 16:51:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:51:30.742232 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248fd8d-f21b-4bc9-94d5-ae0b24099d1a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:30.744360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1248fd8d-f21b-4bc9-94d5-ae0b24099d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:30.917215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248fd8d-f21b-4bc9-94d5-ae0b24099d1a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:31.010594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550d44e-265c-402d-b0b6-2125538f953b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:31.011365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550d44e-265c-402d-b0b6-2125538f953b service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:31.011706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550d44e-265c-402d-b0b6-2125538f953b service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:31.012036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550d44e-265c-402d-b0b6-2125538f953b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:31.013880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c550d44e-265c-402d-b0b6-2125538f953b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:31.074629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:31.147342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550d44e-265c-402d-b0b6-2125538f953b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:31.150707 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550d44e-265c-402d-b0b6-2125538f953b service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:31.185608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 659/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:33.601763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7ee04-c087-454c-9e51-c3f648341d6b None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:33.602346 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7ee04-c087-454c-9e51-c3f648341d6b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:33.602346 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7ee04-c087-454c-9e51-c3f648341d6b None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:33.603857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 653/1312] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:33.609921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45bbce-a653-474d-be07-214d3ca219fd None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:33.610394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45bbce-a653-474d-be07-214d3ca219fd None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:33.610503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45bbce-a653-474d-be07-214d3ca219fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:33.690150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-3d45bbce-a653-474d-be07-214d3ca219fd None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:33.691648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3d45bbce-a653-474d-be07-214d3ca219fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:33.796152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 660/1313] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:33.855391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aef8f5-eed2-419c-8a6a-aa27b65ced4d None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:33.855830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aef8f5-eed2-419c-8a6a-aa27b65ced4d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:33.856163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aef8f5-eed2-419c-8a6a-aa27b65ced4d None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:33.857645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 654/1314] 10.222.0.57 () {54 vars in 965 bytes} [Mon Aug 8 16:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:33.865820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448ef92-f695-4cfa-a72c-28b9d3a96935 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:33.866200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448ef92-f695-4cfa-a72c-28b9d3a96935 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:33.866537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448ef92-f695-4cfa-a72c-28b9d3a96935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:33.879310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:33.955959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d448ef92-f695-4cfa-a72c-28b9d3a96935 None None] MFA Rules not processed for user `f50ee57e2418417cbe68915ca3f00f76`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:33.957208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d448ef92-f695-4cfa-a72c-28b9d3a96935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:34.042769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 661/1315] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Aug 8 16:51:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:34.051922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aef8f5-eed2-419c-8a6a-aa27b65ced4d None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:34.053461 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b1aef8f5-eed2-419c-8a6a-aa27b65ced4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:34.166444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aef8f5-eed2-419c-8a6a-aa27b65ced4d None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:34.245210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:34.247604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2da5d-1b9c-427c-b9db-de45ee981942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:34.248289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2da5d-1b9c-427c-b9db-de45ee981942 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:34.249062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2da5d-1b9c-427c-b9db-de45ee981942 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:34.249335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2da5d-1b9c-427c-b9db-de45ee981942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:34.251074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cdf2da5d-1b9c-427c-b9db-de45ee981942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:34.383882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2da5d-1b9c-427c-b9db-de45ee981942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:34.385656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2da5d-1b9c-427c-b9db-de45ee981942 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:34.411489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 655/1316] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:37.271307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dbc65-73e0-40ae-a693-1cdb119dab8b None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:37.271860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dbc65-73e0-40ae-a693-1cdb119dab8b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:37.271860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dbc65-73e0-40ae-a693-1cdb119dab8b None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:37.273193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 662/1317] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:37.278657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3155-3906-4952-841e-3fe0b2da8b06 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:37.279159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3155-3906-4952-841e-3fe0b2da8b06 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:37.279271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3155-3906-4952-841e-3fe0b2da8b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:37.349334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-909f3155-3906-4952-841e-3fe0b2da8b06 None None] MFA Rules not processed for user `3b9f97788d554a69b9c325c4e2da6890`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:37.350594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-909f3155-3906-4952-841e-3fe0b2da8b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:37.403738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:37.452752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 656/1318] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:37.492997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0dbc65-73e0-40ae-a693-1cdb119dab8b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:37.494595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dd0dbc65-73e0-40ae-a693-1cdb119dab8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:37.596181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0dbc65-73e0-40ae-a693-1cdb119dab8b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:37.599129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:37.672862 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bce7a-a684-4cef-b36e-df8c512f6738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:37.673477 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bce7a-a684-4cef-b36e-df8c512f6738 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:37.673752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bce7a-a684-4cef-b36e-df8c512f6738 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:37.674055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bce7a-a684-4cef-b36e-df8c512f6738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:37.675706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9a3bce7a-a684-4cef-b36e-df8c512f6738 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:37.801194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bce7a-a684-4cef-b36e-df8c512f6738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:37.802708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bce7a-a684-4cef-b36e-df8c512f6738 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:37.823962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 663/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:40.843125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8041f6-7d45-4441-8889-dab936a3d932 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:40.843630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8041f6-7d45-4441-8889-dab936a3d932 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:40.843630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8041f6-7d45-4441-8889-dab936a3d932 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:40.844924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 657/1320] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:40.849989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad70d9a-cc88-42d9-84b4-b76750e470c8 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:40.850412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad70d9a-cc88-42d9-84b4-b76750e470c8 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:40.850536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad70d9a-cc88-42d9-84b4-b76750e470c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:40.892453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:40.916126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-aad70d9a-cc88-42d9-84b4-b76750e470c8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:40.917314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-aad70d9a-cc88-42d9-84b4-b76750e470c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:41.015003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 664/1321] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:41.054518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8041f6-7d45-4441-8889-dab936a3d932 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:41.056067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3e8041f6-7d45-4441-8889-dab936a3d932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:41.156351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8041f6-7d45-4441-8889-dab936a3d932 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:41.227540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8f960-03db-47fa-b0ee-c9341bd45ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:41.228142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8f960-03db-47fa-b0ee-c9341bd45ffc service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:41.228411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8f960-03db-47fa-b0ee-c9341bd45ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:41.228671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8f960-03db-47fa-b0ee-c9341bd45ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:41.230287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-aff8f960-03db-47fa-b0ee-c9341bd45ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:41.239142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:41.336304 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8f960-03db-47fa-b0ee-c9341bd45ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:41.337732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8f960-03db-47fa-b0ee-c9341bd45ffc service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:41.355644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 658/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:44.586961 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676d870-950d-425b-9e45-a523b52a0c99 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.587482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676d870-950d-425b-9e45-a523b52a0c99 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.588033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676d870-950d-425b-9e45-a523b52a0c99 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.588962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 665/1323] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:44.594452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a10e7-e091-4d72-93b9-cc89b11ec0c6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:44.596494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a10e7-e091-4d72-93b9-cc89b11ec0c6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:44.596666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a10e7-e091-4d72-93b9-cc89b11ec0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:44.663601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-7f3a10e7-e091-4d72-93b9-cc89b11ec0c6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:44.664756 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7f3a10e7-e091-4d72-93b9-cc89b11ec0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:44.726577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:44.765947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 659/1324] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:44.802092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676d870-950d-425b-9e45-a523b52a0c99 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:44.804059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5676d870-950d-425b-9e45-a523b52a0c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:44.923186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676d870-950d-425b-9e45-a523b52a0c99 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:44.937318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:45.027540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e582a-0f4a-4b74-8e8b-2090a6999dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:45.028167 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e582a-0f4a-4b74-8e8b-2090a6999dea service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:45.028455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e582a-0f4a-4b74-8e8b-2090a6999dea service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:45.028727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e582a-0f4a-4b74-8e8b-2090a6999dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:45.030582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-671e582a-0f4a-4b74-8e8b-2090a6999dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:45.146458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e582a-0f4a-4b74-8e8b-2090a6999dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:45.147893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e582a-0f4a-4b74-8e8b-2090a6999dea service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:45.174933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 666/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:47.296866 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb312664-adf7-47a7-a3f6-a915c541c709 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.297431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb312664-adf7-47a7-a3f6-a915c541c709 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.297528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb312664-adf7-47a7-a3f6-a915c541c709 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.298937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 660/1326] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:47.305158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae0739-2781-45d8-8272-fcc58ce868b9 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.305972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae0739-2781-45d8-8272-fcc58ce868b9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.305972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae0739-2781-45d8-8272-fcc58ce868b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.356395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:47.376925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-99ae0739-2781-45d8-8272-fcc58ce868b9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:47.378162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-99ae0739-2781-45d8-8272-fcc58ce868b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:47.474881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 667/1327] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:47.511706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb312664-adf7-47a7-a3f6-a915c541c709 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:47.513380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-bb312664-adf7-47a7-a3f6-a915c541c709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:47.638822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb312664-adf7-47a7-a3f6-a915c541c709 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:47.718312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859fc52-fd62-4f09-b0bf-5563e6d4a289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:47.718930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859fc52-fd62-4f09-b0bf-5563e6d4a289 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:47.719207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859fc52-fd62-4f09-b0bf-5563e6d4a289 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:47.719484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859fc52-fd62-4f09-b0bf-5563e6d4a289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:47.721093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3859fc52-fd62-4f09-b0bf-5563e6d4a289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:47.739928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:47.839727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859fc52-fd62-4f09-b0bf-5563e6d4a289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:47.841199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859fc52-fd62-4f09-b0bf-5563e6d4a289 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:47.862685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 661/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:50.022625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b49bd-1b18-46fc-843c-c8d76b5fa144 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:50.023426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b49bd-1b18-46fc-843c-c8d76b5fa144 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:50.023426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b49bd-1b18-46fc-843c-c8d76b5fa144 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:50.025013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 668/1329] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:50.031467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d3833-8958-48c6-ba53-9fec3222260b None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:50.031992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d3833-8958-48c6-ba53-9fec3222260b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:50.032232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d3833-8958-48c6-ba53-9fec3222260b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:50.103266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a12d3833-8958-48c6-ba53-9fec3222260b None None] MFA Rules not processed for user `3b9f97788d554a69b9c325c4e2da6890`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:50.104481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a12d3833-8958-48c6-ba53-9fec3222260b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:50.179168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:50.210329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 662/1330] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:50.248316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b49bd-1b18-46fc-843c-c8d76b5fa144 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:50.249896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1d8b49bd-1b18-46fc-843c-c8d76b5fa144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:50.355044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b49bd-1b18-46fc-843c-c8d76b5fa144 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:50.375097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:50.433378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9e71f-de0a-43f1-81b7-34d5049e119f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:50.434030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9e71f-de0a-43f1-81b7-34d5049e119f service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:50.434305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9e71f-de0a-43f1-81b7-34d5049e119f service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:50.434572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9e71f-de0a-43f1-81b7-34d5049e119f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:50.436321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-fcd9e71f-de0a-43f1-81b7-34d5049e119f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:50.552763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9e71f-de0a-43f1-81b7-34d5049e119f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:50.554294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9e71f-de0a-43f1-81b7-34d5049e119f service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:50.576413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 669/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:53.244432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4b2ad-1148-49fa-a532-e50917a55416 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.245009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4b2ad-1148-49fa-a532-e50917a55416 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.245009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4b2ad-1148-49fa-a532-e50917a55416 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.246390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 663/1332] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:53.251644 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cfc3d-92aa-41d5-8281-65a33682be9a None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.252077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cfc3d-92aa-41d5-8281-65a33682be9a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.252166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cfc3d-92aa-41d5-8281-65a33682be9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.316682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:53.320741 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2a3cfc3d-92aa-41d5-8281-65a33682be9a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:53.322113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2a3cfc3d-92aa-41d5-8281-65a33682be9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:53.422359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 670/1333] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:53.459525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4b2ad-1148-49fa-a532-e50917a55416 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:53.461403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3ae4b2ad-1148-49fa-a532-e50917a55416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:53.569005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4b2ad-1148-49fa-a532-e50917a55416 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:53.641704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936142b-f9fc-47a9-9905-0696cb3d9029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:53.642326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936142b-f9fc-47a9-9905-0696cb3d9029 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:53.642597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936142b-f9fc-47a9-9905-0696cb3d9029 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:53.642861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936142b-f9fc-47a9-9905-0696cb3d9029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:53.644438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d936142b-f9fc-47a9-9905-0696cb3d9029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:53.672233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:53.755556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936142b-f9fc-47a9-9905-0696cb3d9029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:53.756985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936142b-f9fc-47a9-9905-0696cb3d9029 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:53.777768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 664/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:55.864901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45c84e-32df-47f2-b1ed-f7d9a7ec20a5 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:55.865417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45c84e-32df-47f2-b1ed-f7d9a7ec20a5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:55.865755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45c84e-32df-47f2-b1ed-f7d9a7ec20a5 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:55.868324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 671/1335] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:55.874049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ccdac-d15e-4da8-b10c-12a75b733dae None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:55.874482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ccdac-d15e-4da8-b10c-12a75b733dae None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:55.874683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ccdac-d15e-4da8-b10c-12a75b733dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:55.946301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5e6ccdac-d15e-4da8-b10c-12a75b733dae None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:55.947507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5e6ccdac-d15e-4da8-b10c-12a75b733dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:56.026392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:56.049419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 665/1336] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:56.092020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45c84e-32df-47f2-b1ed-f7d9a7ec20a5 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:56.093613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0b45c84e-32df-47f2-b1ed-f7d9a7ec20a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:56.198088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45c84e-32df-47f2-b1ed-f7d9a7ec20a5 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:56.227520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:56.275075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b0795-c0ef-478b-b1c2-0aad4ef9e298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:56.275693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b0795-c0ef-478b-b1c2-0aad4ef9e298 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:56.275969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b0795-c0ef-478b-b1c2-0aad4ef9e298 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:56.276235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b0795-c0ef-478b-b1c2-0aad4ef9e298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:56.277905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5f8b0795-c0ef-478b-b1c2-0aad4ef9e298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:56.389317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b0795-c0ef-478b-b1c2-0aad4ef9e298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:56.390804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b0795-c0ef-478b-b1c2-0aad4ef9e298 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:56.408908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 672/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:51:59.066761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fe90c-9936-478d-b723-5d469e522f23 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:59.067369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fe90c-9936-478d-b723-5d469e522f23 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:59.067369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fe90c-9936-478d-b723-5d469e522f23 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:59.068665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 666/1338] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:51:59.074035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06c77a-6fcb-4790-a4ee-34deff175d39 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:59.074546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06c77a-6fcb-4790-a4ee-34deff175d39 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:59.074650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06c77a-6fcb-4790-a4ee-34deff175d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:59.142932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-5f06c77a-6fcb-4790-a4ee-34deff175d39 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:51:59.144142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5f06c77a-6fcb-4790-a4ee-34deff175d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:59.152590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:59.240718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 673/1339] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:51:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:51:59.278492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fe90c-9936-478d-b723-5d469e522f23 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:51:59.280146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-165fe90c-9936-478d-b723-5d469e522f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:59.390577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fe90c-9936-478d-b723-5d469e522f23 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:51:59.467902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec88004-861f-41d5-b900-8a54b776ea7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:51:59.468541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88004-861f-41d5-b900-8a54b776ea7f service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:51:59.468817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88004-861f-41d5-b900-8a54b776ea7f service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:51:59.469087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88004-861f-41d5-b900-8a54b776ea7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:51:59.470773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0ec88004-861f-41d5-b900-8a54b776ea7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:51:59.509310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:51:59.585243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec88004-861f-41d5-b900-8a54b776ea7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:51:59.586726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec88004-861f-41d5-b900-8a54b776ea7f service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:51:59.606949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 667/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:51:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:04.102094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cd210-8877-4d57-ba29-f205d0287555 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:04.102649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cd210-8877-4d57-ba29-f205d0287555 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:04.102649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cd210-8877-4d57-ba29-f205d0287555 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:04.103977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 674/1341] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:04.111031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a08c2-1c4a-4aa8-b32b-de0892e7b17a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:04.111551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a08c2-1c4a-4aa8-b32b-de0892e7b17a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:04.111881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a08c2-1c4a-4aa8-b32b-de0892e7b17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:04.214230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-da2a08c2-1c4a-4aa8-b32b-de0892e7b17a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:04.215372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-da2a08c2-1c4a-4aa8-b32b-de0892e7b17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:04.317216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:04.338890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 668/1342] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:04.378897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cd210-8877-4d57-ba29-f205d0287555 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:04.381366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5a1cd210-8877-4d57-ba29-f205d0287555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:04.503837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cd210-8877-4d57-ba29-f205d0287555 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:04.542863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:04.582967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f08c17-3e51-4c29-9256-94388fb5b035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:04.583590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f08c17-3e51-4c29-9256-94388fb5b035 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:04.583862 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f08c17-3e51-4c29-9256-94388fb5b035 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:04.584124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f08c17-3e51-4c29-9256-94388fb5b035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:04.585751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a9f08c17-3e51-4c29-9256-94388fb5b035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:04.709013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f08c17-3e51-4c29-9256-94388fb5b035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:04.710663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f08c17-3e51-4c29-9256-94388fb5b035 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:04.730582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 675/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:07.796506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccca2d-4a44-46b4-8557-b43219eec0bb None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:07.797344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccca2d-4a44-46b4-8557-b43219eec0bb None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:07.797344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccca2d-4a44-46b4-8557-b43219eec0bb None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:07.798441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 669/1344] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:07.804904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf82ec-d07c-4a12-832a-2ebf1211ed7d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:07.805420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf82ec-d07c-4a12-832a-2ebf1211ed7d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:07.805915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf82ec-d07c-4a12-832a-2ebf1211ed7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:07.882327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9ddf82ec-d07c-4a12-832a-2ebf1211ed7d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:07.883472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9ddf82ec-d07c-4a12-832a-2ebf1211ed7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:07.902683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:07.987112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 676/1345] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:08.024720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccca2d-4a44-46b4-8557-b43219eec0bb None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:08.026586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-55ccca2d-4a44-46b4-8557-b43219eec0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:08.159248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccca2d-4a44-46b4-8557-b43219eec0bb None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:08.238839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fcef0-5ea1-4d90-a828-f6d7fd154b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:08.239440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fcef0-5ea1-4d90-a828-f6d7fd154b38 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:08.239716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fcef0-5ea1-4d90-a828-f6d7fd154b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:08.239986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fcef0-5ea1-4d90-a828-f6d7fd154b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:08.241603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-536fcef0-5ea1-4d90-a828-f6d7fd154b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:08.290506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:08.368061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fcef0-5ea1-4d90-a828-f6d7fd154b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:08.369501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fcef0-5ea1-4d90-a828-f6d7fd154b38 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:08.389875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 670/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:52:12.793886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499f561-aeb3-4ee0-b574-a34e1581e0c4 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.794410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499f561-aeb3-4ee0-b574-a34e1581e0c4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.794410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499f561-aeb3-4ee0-b574-a34e1581e0c4 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.796562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 677/1347] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:12.802831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0855cb3-0f50-496a-a062-b8ca7213056a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:12.803383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0855cb3-0f50-496a-a062-b8ca7213056a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:12.803611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0855cb3-0f50-496a-a062-b8ca7213056a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:12.903216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a0855cb3-0f50-496a-a062-b8ca7213056a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:12.905822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a0855cb3-0f50-496a-a062-b8ca7213056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:13.051677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:13.054694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 671/1348] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:13.106314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499f561-aeb3-4ee0-b574-a34e1581e0c4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:13.107981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e499f561-aeb3-4ee0-b574-a34e1581e0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:13.248411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499f561-aeb3-4ee0-b574-a34e1581e0c4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:13.317936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:13.351833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92eb7d-41b2-413a-ac83-1742fab5f046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:13.352454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92eb7d-41b2-413a-ac83-1742fab5f046 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:13.352734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92eb7d-41b2-413a-ac83-1742fab5f046 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:13.352998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92eb7d-41b2-413a-ac83-1742fab5f046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:13.354950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5f92eb7d-41b2-413a-ac83-1742fab5f046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:13.497055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92eb7d-41b2-413a-ac83-1742fab5f046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:13.498588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92eb7d-41b2-413a-ac83-1742fab5f046 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:13.527068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 678/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:19.069959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4f35e-b269-4c7a-a3b4-b5fe1fb9c895 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:19.070503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4f35e-b269-4c7a-a3b4-b5fe1fb9c895 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:19.070595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4f35e-b269-4c7a-a3b4-b5fe1fb9c895 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:19.072381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 672/1350] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 16:52:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:52:19.078342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b5836-caac-4156-9e09-d3160ce9c599 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:19.078964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b5836-caac-4156-9e09-d3160ce9c599 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:19.078964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b5836-caac-4156-9e09-d3160ce9c599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:19.208786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dd6b5836-caac-4156-9e09-d3160ce9c599 None None] MFA Rules not processed for user `5853f9b60cf34380b218a5cda94f587d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:19.210457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dd6b5836-caac-4156-9e09-d3160ce9c599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:19.236874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:19.433286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 679/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 16:52:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:24.560071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940d92-18e7-4621-9a69-1f5c33a0f262 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:24.560648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940d92-18e7-4621-9a69-1f5c33a0f262 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:24.560648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940d92-18e7-4621-9a69-1f5c33a0f262 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:24.562248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 673/1352] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 16:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:24.567418 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14208-c5de-42b0-8dbe-bbb8269e1f80 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:24.567887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14208-c5de-42b0-8dbe-bbb8269e1f80 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:24.567993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14208-c5de-42b0-8dbe-bbb8269e1f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:24.657181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-8fb14208-c5de-42b0-8dbe-bbb8269e1f80 None None] MFA Rules not processed for user `5853f9b60cf34380b218a5cda94f587d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:24.658426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8fb14208-c5de-42b0-8dbe-bbb8269e1f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:24.743396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:24.800677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 680/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 16:52:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:26.322578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa71b18-dafd-4bed-9d61-8d58d0fe3901 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:26.323251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa71b18-dafd-4bed-9d61-8d58d0fe3901 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:26.323251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa71b18-dafd-4bed-9d61-8d58d0fe3901 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:26.324598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 674/1354] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 16:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:26.332170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ff714-24fb-4ddd-a3a5-bee4030e89d7 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:26.332681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ff714-24fb-4ddd-a3a5-bee4030e89d7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:26.332758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ff714-24fb-4ddd-a3a5-bee4030e89d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:26.414950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-be1ff714-24fb-4ddd-a3a5-bee4030e89d7 None None] MFA Rules not processed for user `5853f9b60cf34380b218a5cda94f587d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:26.416203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-be1ff714-24fb-4ddd-a3a5-bee4030e89d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:26.560899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 681/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 16:52:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:30.125112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8cdec-b0ce-422f-9f1b-1dcaea89c98f None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.125112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8cdec-b0ce-422f-9f1b-1dcaea89c98f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.125112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8cdec-b0ce-422f-9f1b-1dcaea89c98f None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:30.127261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 675/1356] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:30.136718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0ea70-248d-4523-834d-fb319ac593a1 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.137584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0ea70-248d-4523-834d-fb319ac593a1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.137584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0ea70-248d-4523-834d-fb319ac593a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:30.196756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:30.255551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-06f0ea70-248d-4523-834d-fb319ac593a1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:30.257372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-06f0ea70-248d-4523-834d-fb319ac593a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:30.397928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 682/1357] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:30.470920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e3a2a-2e86-4b4f-ac62-c1ced91895ff None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.471214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e3a2a-2e86-4b4f-ac62-c1ced91895ff None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.472133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e3a2a-2e86-4b4f-ac62-c1ced91895ff None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:30.472824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 676/1358] 10.222.0.57 () {54 vars in 962 bytes} [Mon Aug 8 16:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:30.486145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82958e2-2f30-43de-905f-97aa94abd1a6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:30.486508 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82958e2-2f30-43de-905f-97aa94abd1a6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:30.486692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82958e2-2f30-43de-905f-97aa94abd1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:30.620630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d82958e2-2f30-43de-905f-97aa94abd1a6 None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:30.622354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d82958e2-2f30-43de-905f-97aa94abd1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:30.702346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:30.809258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 683/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Aug 8 16:52:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:30.818086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e3a2a-2e86-4b4f-ac62-c1ced91895ff None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:30.844154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2f5e3a2a-2e86-4b4f-ac62-c1ced91895ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:31.093841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e3a2a-2e86-4b4f-ac62-c1ced91895ff None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:31.202603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:31.277746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d19803-b6da-4f37-933a-a56ce9e5a7eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:31.279305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d19803-b6da-4f37-933a-a56ce9e5a7eb service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:31.279804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d19803-b6da-4f37-933a-a56ce9e5a7eb service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:31.280289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d19803-b6da-4f37-933a-a56ce9e5a7eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:31.283338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e7d19803-b6da-4f37-933a-a56ce9e5a7eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:31.410481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d19803-b6da-4f37-933a-a56ce9e5a7eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:31.414211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d19803-b6da-4f37-933a-a56ce9e5a7eb service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:31.447112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 677/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:33.946694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c1a0a-8bca-40c5-8584-4a810c0ccbb1 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:33.947194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c1a0a-8bca-40c5-8584-4a810c0ccbb1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:33.947643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c1a0a-8bca-40c5-8584-4a810c0ccbb1 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:33.948864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 684/1361] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:33.954268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e547e-677b-4f5c-af4e-d2496d9896aa None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:33.954697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e547e-677b-4f5c-af4e-d2496d9896aa None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:33.954881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e547e-677b-4f5c-af4e-d2496d9896aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:34.039733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-af0e547e-677b-4f5c-af4e-d2496d9896aa None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:34.040499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-af0e547e-677b-4f5c-af4e-d2496d9896aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:34.086040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:34.165921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 678/1362] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:34.210279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c1a0a-8bca-40c5-8584-4a810c0ccbb1 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:34.211924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-212c1a0a-8bca-40c5-8584-4a810c0ccbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:34.384333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c1a0a-8bca-40c5-8584-4a810c0ccbb1 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:34.396615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:34.646377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44aa5bc-48df-41b8-818f-f37df107ae9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:34.646995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aa5bc-48df-41b8-818f-f37df107ae9a service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:34.647269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aa5bc-48df-41b8-818f-f37df107ae9a service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:34.647544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aa5bc-48df-41b8-818f-f37df107ae9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:34.649183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e44aa5bc-48df-41b8-818f-f37df107ae9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:34.760940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44aa5bc-48df-41b8-818f-f37df107ae9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:34.762592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44aa5bc-48df-41b8-818f-f37df107ae9a service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:34.784342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 685/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:52:37.377123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcec8ad-9b28-4738-a2f5-f2cece60b36c None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:37.378112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcec8ad-9b28-4738-a2f5-f2cece60b36c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:37.378435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcec8ad-9b28-4738-a2f5-f2cece60b36c None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:37.381404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 679/1364] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:37.389616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299951b3-6b3f-4fb1-9c88-e3d54309cd98 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:37.390430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299951b3-6b3f-4fb1-9c88-e3d54309cd98 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:37.390430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299951b3-6b3f-4fb1-9c88-e3d54309cd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:37.411215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:37.485610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-299951b3-6b3f-4fb1-9c88-e3d54309cd98 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:37.487087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-299951b3-6b3f-4fb1-9c88-e3d54309cd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:37.640462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 686/1365] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:37.691075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640be2e-a77a-49f7-9cde-4e0c05c5bbaa None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:37.691753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640be2e-a77a-49f7-9cde-4e0c05c5bbaa None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:37.692134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640be2e-a77a-49f7-9cde-4e0c05c5bbaa None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:37.693821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 680/1366] 10.222.0.57 () {54 vars in 962 bytes} [Mon Aug 8 16:52:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:52:37.703429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbcf5b-d397-4aa9-b0b2-8658dd79534d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:37.703763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbcf5b-d397-4aa9-b0b2-8658dd79534d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:37.703981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbcf5b-d397-4aa9-b0b2-8658dd79534d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:37.837125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dcdbcf5b-d397-4aa9-b0b2-8658dd79534d None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:37.838532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dcdbcf5b-d397-4aa9-b0b2-8658dd79534d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:37.878400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:37.992300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 687/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Aug 8 16:52:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:37.994420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640be2e-a77a-49f7-9cde-4e0c05c5bbaa None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:37.996297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c640be2e-a77a-49f7-9cde-4e0c05c5bbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:38.191849 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640be2e-a77a-49f7-9cde-4e0c05c5bbaa None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:38.222610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:38.343334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568d5a9-1694-4caa-a414-43d9fae410ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:38.344379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568d5a9-1694-4caa-a414-43d9fae410ca service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:38.344840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568d5a9-1694-4caa-a414-43d9fae410ca service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:38.345210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568d5a9-1694-4caa-a414-43d9fae410ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:38.347157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7568d5a9-1694-4caa-a414-43d9fae410ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:38.467240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568d5a9-1694-4caa-a414-43d9fae410ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:38.469027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568d5a9-1694-4caa-a414-43d9fae410ca service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:38.489684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 681/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:39.190241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137e408-06c2-4a11-ba50-71501743d989 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:39.191184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137e408-06c2-4a11-ba50-71501743d989 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:39.191632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137e408-06c2-4a11-ba50-71501743d989 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:39.197106 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 688/1369] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:39.199254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddede3-9f9a-4e3d-ba6e-74c38378034e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:39.199766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddede3-9f9a-4e3d-ba6e-74c38378034e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:39.200232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddede3-9f9a-4e3d-ba6e-74c38378034e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:39.235979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:39.277205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-95ddede3-9f9a-4e3d-ba6e-74c38378034e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:39.279478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-95ddede3-9f9a-4e3d-ba6e-74c38378034e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:39.392428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 682/1370] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:39.739361 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0822a93-f5fb-4abe-b135-df81e97795a2 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:39.739671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0822a93-f5fb-4abe-b135-df81e97795a2 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:39.739907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0822a93-f5fb-4abe-b135-df81e97795a2 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:39.741458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 689/1371] 10.222.0.57 () {54 vars in 964 bytes} [Mon Aug 8 16:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:39.746854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac04241-c568-4cdf-b282-d249b97d73a5 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:39.747789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac04241-c568-4cdf-b282-d249b97d73a5 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:39.748278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac04241-c568-4cdf-b282-d249b97d73a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:39.836099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-eac04241-c568-4cdf-b282-d249b97d73a5 None None] MFA Rules not processed for user `e860aad24688491c9a89ddfb35922eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:39.837338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-eac04241-c568-4cdf-b282-d249b97d73a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:39.897997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:39.946274 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 683/1372] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Aug 8 16:52:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:39.947377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0822a93-f5fb-4abe-b135-df81e97795a2 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:39.948968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e0822a93-f5fb-4abe-b135-df81e97795a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:40.117321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0822a93-f5fb-4abe-b135-df81e97795a2 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:40.195496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:40.242216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badcd48-09b9-4a85-b7e5-45aae55192a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:40.242910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badcd48-09b9-4a85-b7e5-45aae55192a8 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:40.243084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badcd48-09b9-4a85-b7e5-45aae55192a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:40.243347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badcd48-09b9-4a85-b7e5-45aae55192a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:40.245104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3badcd48-09b9-4a85-b7e5-45aae55192a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:40.374933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3badcd48-09b9-4a85-b7e5-45aae55192a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:40.374933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3badcd48-09b9-4a85-b7e5-45aae55192a8 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:40.405095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 690/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:40.921746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d21a7-de74-4aa9-8e39-75f1454fc592 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:40.922379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d21a7-de74-4aa9-8e39-75f1454fc592 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:40.922379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d21a7-de74-4aa9-8e39-75f1454fc592 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:40.923691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 684/1374] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:40.929404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88599811-4080-4aca-ac21-f12d5af573ca None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:40.929739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88599811-4080-4aca-ac21-f12d5af573ca None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:40.930018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88599811-4080-4aca-ac21-f12d5af573ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:41.004558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-88599811-4080-4aca-ac21-f12d5af573ca None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:41.005921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-88599811-4080-4aca-ac21-f12d5af573ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:41.030121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:41.120560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 691/1375] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:41.166514 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d21a7-de74-4aa9-8e39-75f1454fc592 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:41.168103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c27d21a7-de74-4aa9-8e39-75f1454fc592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:41.327941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d21a7-de74-4aa9-8e39-75f1454fc592 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:41.380428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:41.442555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81720374-c825-4f91-aa22-ede2499ddc32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:41.443157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81720374-c825-4f91-aa22-ede2499ddc32 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:41.443434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81720374-c825-4f91-aa22-ede2499ddc32 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:41.443702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81720374-c825-4f91-aa22-ede2499ddc32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:41.445291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-81720374-c825-4f91-aa22-ede2499ddc32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:41.549481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81720374-c825-4f91-aa22-ede2499ddc32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:41.551232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81720374-c825-4f91-aa22-ede2499ddc32 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:41.573668 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 685/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:52:43.932939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d26f5-4a7b-4f47-b095-59fd19e5d149 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:43.941717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d26f5-4a7b-4f47-b095-59fd19e5d149 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:43.942177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d26f5-4a7b-4f47-b095-59fd19e5d149 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:43.943643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 692/1377] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:43.952499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb987a42-3c98-436c-9928-49f4ca9b345c None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:43.953013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb987a42-3c98-436c-9928-49f4ca9b345c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:43.953281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb987a42-3c98-436c-9928-49f4ca9b345c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:44.075334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-fb987a42-3c98-436c-9928-49f4ca9b345c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:44.076594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fb987a42-3c98-436c-9928-49f4ca9b345c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:44.085207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:44.192075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 686/1378] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:44.240891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d26f5-4a7b-4f47-b095-59fd19e5d149 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:44.242440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e26d26f5-4a7b-4f47-b095-59fd19e5d149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:44.494231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d26f5-4a7b-4f47-b095-59fd19e5d149 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:44.501896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:44.719332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f048eab-70de-4d2e-a710-95a050482fb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:44.719941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f048eab-70de-4d2e-a710-95a050482fb8 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:44.720211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f048eab-70de-4d2e-a710-95a050482fb8 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:44.720469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f048eab-70de-4d2e-a710-95a050482fb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:44.722157 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3f048eab-70de-4d2e-a710-95a050482fb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:44.867454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f048eab-70de-4d2e-a710-95a050482fb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:44.868943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f048eab-70de-4d2e-a710-95a050482fb8 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:44.890939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 693/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:47.318084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565605e-cc5c-485d-a9a6-4098c89de8fc None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:47.318673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565605e-cc5c-485d-a9a6-4098c89de8fc None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:47.318673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565605e-cc5c-485d-a9a6-4098c89de8fc None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:47.320150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 687/1380] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:47.325941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fc3d1-fcdf-4d99-aa05-c122ebfd6791 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:47.326424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fc3d1-fcdf-4d99-aa05-c122ebfd6791 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:47.326532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fc3d1-fcdf-4d99-aa05-c122ebfd6791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:47.338551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:47.411711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-7c7fc3d1-fcdf-4d99-aa05-c122ebfd6791 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:47.412810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7c7fc3d1-fcdf-4d99-aa05-c122ebfd6791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:47.537633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 694/1381] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:47.584566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565605e-cc5c-485d-a9a6-4098c89de8fc None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:47.585516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f565605e-cc5c-485d-a9a6-4098c89de8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:47.766935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:47.786687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565605e-cc5c-485d-a9a6-4098c89de8fc None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:47.992240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fb587-4d89-4f0b-85a3-aa5a36c5aff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:47.993285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fb587-4d89-4f0b-85a3-aa5a36c5aff0 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:47.993845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fb587-4d89-4f0b-85a3-aa5a36c5aff0 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:47.994328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fb587-4d89-4f0b-85a3-aa5a36c5aff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:47.997980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c16fb587-4d89-4f0b-85a3-aa5a36c5aff0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:48.166990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fb587-4d89-4f0b-85a3-aa5a36c5aff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:48.170133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fb587-4d89-4f0b-85a3-aa5a36c5aff0 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:48.184757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:48.201236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 688/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:51.037762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d3020-2da7-41a0-9630-c7a4257df8e7 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:51.038498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d3020-2da7-41a0-9630-c7a4257df8e7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:51.038498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d3020-2da7-41a0-9630-c7a4257df8e7 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:51.039718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 695/1383] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:51.045551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f22b19-0f67-45ca-8a6f-98b5d4493a4e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:51.046322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f22b19-0f67-45ca-8a6f-98b5d4493a4e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:51.046322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f22b19-0f67-45ca-8a6f-98b5d4493a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:51.134213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-57f22b19-0f67-45ca-8a6f-98b5d4493a4e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:51.136147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-57f22b19-0f67-45ca-8a6f-98b5d4493a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:51.269174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 689/1384] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:51.315154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d3020-2da7-41a0-9630-c7a4257df8e7 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:51.316783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ab8d3020-2da7-41a0-9630-c7a4257df8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:51.428037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:51.504387 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d3020-2da7-41a0-9630-c7a4257df8e7 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:51.644477 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20887d40-5e5e-4edc-98f8-4f32ed4eddad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:51.645079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887d40-5e5e-4edc-98f8-4f32ed4eddad service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:51.645352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887d40-5e5e-4edc-98f8-4f32ed4eddad service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:51.645628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887d40-5e5e-4edc-98f8-4f32ed4eddad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:51.647300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-20887d40-5e5e-4edc-98f8-4f32ed4eddad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:51.712174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:51.764808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20887d40-5e5e-4edc-98f8-4f32ed4eddad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:51.766559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20887d40-5e5e-4edc-98f8-4f32ed4eddad service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:51.787626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 696/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:54.098187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fed69d-dd70-453d-a1f9-a6dbd6eefb4e None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:54.098778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fed69d-dd70-453d-a1f9-a6dbd6eefb4e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:54.098778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fed69d-dd70-453d-a1f9-a6dbd6eefb4e None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:54.100105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 690/1386] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:54.105848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611fdc1-71c9-49e3-b8c9-a430510ec852 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:54.106943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611fdc1-71c9-49e3-b8c9-a430510ec852 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:54.106943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611fdc1-71c9-49e3-b8c9-a430510ec852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:54.192095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c611fdc1-71c9-49e3-b8c9-a430510ec852 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:54.193662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c611fdc1-71c9-49e3-b8c9-a430510ec852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:54.324674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 697/1387] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:54.371118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fed69d-dd70-453d-a1f9-a6dbd6eefb4e None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:54.373735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a9fed69d-dd70-453d-a1f9-a6dbd6eefb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:54.490077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:54.602453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fed69d-dd70-453d-a1f9-a6dbd6eefb4e None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:54.771928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff6458-79b9-4dff-9279-6000a8d5c776 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:54.773048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff6458-79b9-4dff-9279-6000a8d5c776 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:54.773539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff6458-79b9-4dff-9279-6000a8d5c776 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:54.773991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff6458-79b9-4dff-9279-6000a8d5c776 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:54.776083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-67ff6458-79b9-4dff-9279-6000a8d5c776 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:54.827334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:54.883402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff6458-79b9-4dff-9279-6000a8d5c776 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:54.886676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff6458-79b9-4dff-9279-6000a8d5c776 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:54.919524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 691/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:56.772465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03509c73-6dc5-4a01-b95d-0a001df189a7 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:56.773056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03509c73-6dc5-4a01-b95d-0a001df189a7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:56.773056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03509c73-6dc5-4a01-b95d-0a001df189a7 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:56.774392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 698/1389] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:52:56.780267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285270a-ecb5-4a86-977a-a7861cf9df58 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:56.780778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285270a-ecb5-4a86-977a-a7861cf9df58 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:56.780887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285270a-ecb5-4a86-977a-a7861cf9df58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:56.870752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d285270a-ecb5-4a86-977a-a7861cf9df58 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:56.872616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d285270a-ecb5-4a86-977a-a7861cf9df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:56.989741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 692/1390] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:57.047377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5e5ee-747b-4c0e-aae8-44ba4eac8c3f None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:57.047651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5e5ee-747b-4c0e-aae8-44ba4eac8c3f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:57.047924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5e5ee-747b-4c0e-aae8-44ba4eac8c3f None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:57.049269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 699/1391] 10.222.0.57 () {54 vars in 967 bytes} [Mon Aug 8 16:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:52:57.054747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1850-fb5a-4c9e-8a87-cabdeb13741e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:57.055056 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1850-fb5a-4c9e-8a87-cabdeb13741e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:57.055460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1850-fb5a-4c9e-8a87-cabdeb13741e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:57.072832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:57.137930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-aefa1850-fb5a-4c9e-8a87-cabdeb13741e None None] MFA Rules not processed for user `73bfe655c6d34918b21acb8748b9156d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:57.139062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-aefa1850-fb5a-4c9e-8a87-cabdeb13741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:57.247356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 693/1392] 10.222.0.57 () {58 vars in 1064 bytes} [Mon Aug 8 16:52:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:57.256274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5e5ee-747b-4c0e-aae8-44ba4eac8c3f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:57.257817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-38d5e5ee-747b-4c0e-aae8-44ba4eac8c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:57.289030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:57.322263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf7792-1079-4b4b-a38d-fa575a69a6c1 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:57.322659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf7792-1079-4b4b-a38d-fa575a69a6c1 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:57.322995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf7792-1079-4b4b-a38d-fa575a69a6c1 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:57.324548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 694/1393] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:52:57.332193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a809fd-af85-4163-8e6c-ccc9ac36e453 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:57.332484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a809fd-af85-4163-8e6c-ccc9ac36e453 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:57.334240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a809fd-af85-4163-8e6c-ccc9ac36e453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:57.378013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5e5ee-747b-4c0e-aae8-44ba4eac8c3f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:57.416119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-64a809fd-af85-4163-8e6c-ccc9ac36e453 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:57.417645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-64a809fd-af85-4163-8e6c-ccc9ac36e453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:57.477545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df52342-6ef7-4fc0-845b-4114d320c876 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73bfe655c6d34918b21acb8748b9156d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:57.478218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df52342-6ef7-4fc0-845b-4114d320c876 service glance] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:57.478557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df52342-6ef7-4fc0-845b-4114d320c876 service glance] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:57.478855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df52342-6ef7-4fc0-845b-4114d320c876 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:57.481711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9df52342-6ef7-4fc0-845b-4114d320c876 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:57.494118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:57.555235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 695/1394] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:57.593444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:57.601461 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a809fd-af85-4163-8e6c-ccc9ac36e453 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:57.602725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-64a809fd-af85-4163-8e6c-ccc9ac36e453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:57.630725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df52342-6ef7-4fc0-845b-4114d320c876 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:57.632186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df52342-6ef7-4fc0-845b-4114d320c876 service glance] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:57.654562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 700/1395] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:57.737380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a809fd-af85-4163-8e6c-ccc9ac36e453 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:57.829226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:57.901672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197ac72-760a-43ca-b4ea-6c782fcf4716 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:57.902345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197ac72-760a-43ca-b4ea-6c782fcf4716 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:57.902604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197ac72-760a-43ca-b4ea-6c782fcf4716 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:57.902880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197ac72-760a-43ca-b4ea-6c782fcf4716 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:57.905244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b197ac72-760a-43ca-b4ea-6c782fcf4716 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:58.018932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197ac72-760a-43ca-b4ea-6c782fcf4716 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:58.020406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197ac72-760a-43ca-b4ea-6c782fcf4716 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:58.040434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 696/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:52:58.616785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38345263-e9d5-4411-a09b-dd9ca3db4331 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:58.617288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38345263-e9d5-4411-a09b-dd9ca3db4331 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:58.617383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38345263-e9d5-4411-a09b-dd9ca3db4331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:58.716675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-38345263-e9d5-4411-a09b-dd9ca3db4331 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:58.718329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-38345263-e9d5-4411-a09b-dd9ca3db4331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:58.834846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 701/1397] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 16:52:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:52:58.898616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8798c7-0812-47e1-b9a5-1c78d539cb40 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:58.899418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8798c7-0812-47e1-b9a5-1c78d539cb40 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:58.899418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8798c7-0812-47e1-b9a5-1c78d539cb40 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:58.900787 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 697/1398] 10.222.0.57 () {54 vars in 943 bytes} [Mon Aug 8 16:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:52:58.909820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2377b3c-82ea-493f-bd11-dc50d47f0981 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:58.910320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2377b3c-82ea-493f-bd11-dc50d47f0981 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:58.910761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2377b3c-82ea-493f-bd11-dc50d47f0981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:58.997396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:59.027272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f2377b3c-82ea-493f-bd11-dc50d47f0981 None None] MFA Rules not processed for user `92130daa3c3a49c2adc597dc563b2973`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:52:59.029057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f2377b3c-82ea-493f-bd11-dc50d47f0981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:59.181682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 702/1399] 10.222.0.57 () {58 vars in 1040 bytes} [Mon Aug 8 16:52:58 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:52:59.197559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8798c7-0812-47e1-b9a5-1c78d539cb40 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:52:59.200677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1b8798c7-0812-47e1-b9a5-1c78d539cb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:59.304588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:59.387763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8798c7-0812-47e1-b9a5-1c78d539cb40 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:52:59.520301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa825bda-ad4e-4169-812a-bf3d8f9a9409 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:52:59.520948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa825bda-ad4e-4169-812a-bf3d8f9a9409 service swift] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:52:59.521218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa825bda-ad4e-4169-812a-bf3d8f9a9409 service swift] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:52:59.521493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa825bda-ad4e-4169-812a-bf3d8f9a9409 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:52:59.523214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fa825bda-ad4e-4169-812a-bf3d8f9a9409 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:52:59.619166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:52:59.642811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa825bda-ad4e-4169-812a-bf3d8f9a9409 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:52:59.644688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa825bda-ad4e-4169-812a-bf3d8f9a9409 service swift] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:52:59.659288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 698/1400] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 16:52:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 16:53:00.459275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae29dd-c15d-4f15-8250-a94732663418 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:00.459776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae29dd-c15d-4f15-8250-a94732663418 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:00.459930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae29dd-c15d-4f15-8250-a94732663418 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:00.461209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 703/1401] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:00.467879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d4cae-b0f7-4418-b2c9-3e16356853cc None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:00.472637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d4cae-b0f7-4418-b2c9-3e16356853cc None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:00.472637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d4cae-b0f7-4418-b2c9-3e16356853cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:00.540908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-437d4cae-b0f7-4418-b2c9-3e16356853cc None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:00.542173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-437d4cae-b0f7-4418-b2c9-3e16356853cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:00.700323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 699/1402] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:00.759108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae29dd-c15d-4f15-8250-a94732663418 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:00.760773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-96ae29dd-c15d-4f15-8250-a94732663418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:00.917179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:01.017569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae29dd-c15d-4f15-8250-a94732663418 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:01.198200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e977fb2-33e1-46fe-818d-a8c3defdd00e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:01.198646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e977fb2-33e1-46fe-818d-a8c3defdd00e service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:01.198920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e977fb2-33e1-46fe-818d-a8c3defdd00e service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:01.199213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e977fb2-33e1-46fe-818d-a8c3defdd00e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:01.201520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4e977fb2-33e1-46fe-818d-a8c3defdd00e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:01.315336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:01.342204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e977fb2-33e1-46fe-818d-a8c3defdd00e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:01.343821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e977fb2-33e1-46fe-818d-a8c3defdd00e service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:01.374104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 704/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:04.275181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da8978-fc92-4970-99a7-28ef9ebe7fa2 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:04.276369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da8978-fc92-4970-99a7-28ef9ebe7fa2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:04.276369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da8978-fc92-4970-99a7-28ef9ebe7fa2 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:04.278233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 700/1404] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:53:04.283211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b2c0a-5f54-4ad8-9eb4-ae75fe838378 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:04.283845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b2c0a-5f54-4ad8-9eb4-ae75fe838378 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:04.283845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b2c0a-5f54-4ad8-9eb4-ae75fe838378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:04.381655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-6c3b2c0a-5f54-4ad8-9eb4-ae75fe838378 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:04.384262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6c3b2c0a-5f54-4ad8-9eb4-ae75fe838378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:04.540786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 705/1405] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:04.595080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da8978-fc92-4970-99a7-28ef9ebe7fa2 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:04.598051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-61da8978-fc92-4970-99a7-28ef9ebe7fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:04.714909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:04.858374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da8978-fc92-4970-99a7-28ef9ebe7fa2 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:05.100843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fbedd-cafa-46ba-acab-165165bcccee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:05.101958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fbedd-cafa-46ba-acab-165165bcccee service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:05.102523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fbedd-cafa-46ba-acab-165165bcccee service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:05.103316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fbedd-cafa-46ba-acab-165165bcccee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:05.106318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-346fbedd-cafa-46ba-acab-165165bcccee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:05.221633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fbedd-cafa-46ba-acab-165165bcccee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:05.224948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fbedd-cafa-46ba-acab-165165bcccee service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:05.266252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 701/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:06.157354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:08.900910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7915842-e731-4558-8813-8a496b5d1b1b None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:08.901691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7915842-e731-4558-8813-8a496b5d1b1b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:08.901691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7915842-e731-4558-8813-8a496b5d1b1b None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:08.903223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 706/1407] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:53:08.909768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1c669-bc7f-4ee3-a19e-86d0cf561387 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:08.910457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1c669-bc7f-4ee3-a19e-86d0cf561387 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:08.910682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1c669-bc7f-4ee3-a19e-86d0cf561387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:09.003746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-29a1c669-bc7f-4ee3-a19e-86d0cf561387 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:09.005026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-29a1c669-bc7f-4ee3-a19e-86d0cf561387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:09.160718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:09.175421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 702/1408] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:09.249678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7915842-e731-4558-8813-8a496b5d1b1b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:09.252205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e7915842-e731-4558-8813-8a496b5d1b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:09.328263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:09.479241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7915842-e731-4558-8813-8a496b5d1b1b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:09.685643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa875d-59ab-490b-b04a-a8f83046ee83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:09.686596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa875d-59ab-490b-b04a-a8f83046ee83 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:09.686885 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa875d-59ab-490b-b04a-a8f83046ee83 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:09.687122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa875d-59ab-490b-b04a-a8f83046ee83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:09.689003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-faaa875d-59ab-490b-b04a-a8f83046ee83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:09.699272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:09.862148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa875d-59ab-490b-b04a-a8f83046ee83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:09.863971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa875d-59ab-490b-b04a-a8f83046ee83 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:09.913229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 707/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:19.250757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:19.886354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=42 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:25.614260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c9e09-94ac-48db-ae8a-08c838993543 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.614873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c9e09-94ac-48db-ae8a-08c838993543 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.614873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c9e09-94ac-48db-ae8a-08c838993543 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.616147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 703/1410] 10.222.0.57 () {54 vars in 918 bytes} [Mon Aug 8 16:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:25.622923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c6f45-f633-4262-ada0-ba51b29022be None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.623371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c6f45-f633-4262-ada0-ba51b29022be None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.623575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c6f45-f633-4262-ada0-ba51b29022be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.704612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-217c6f45-f633-4262-ada0-ba51b29022be None None] MFA Rules not processed for user `5853f9b60cf34380b218a5cda94f587d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:25.705754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-217c6f45-f633-4262-ada0-ba51b29022be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:25.819823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 708/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Aug 8 16:53:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:25.834772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24df01-037b-4bec-b581-fdfef4d8b0c1 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.835082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24df01-037b-4bec-b581-fdfef4d8b0c1 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.835382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24df01-037b-4bec-b581-fdfef4d8b0c1 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.836757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 704/1412] 10.222.0.57 () {54 vars in 918 bytes} [Mon Aug 8 16:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:25.841658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e37231-6c6e-4453-918b-b2d2a17a62f9 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:25.841976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e37231-6c6e-4453-918b-b2d2a17a62f9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:25.842273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e37231-6c6e-4453-918b-b2d2a17a62f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:25.916958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:25.920549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c0e37231-6c6e-4453-918b-b2d2a17a62f9 None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:25.921809 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c0e37231-6c6e-4453-918b-b2d2a17a62f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:26.041337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 709/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Aug 8 16:53:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:28.610895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ed7bd-04f6-448c-92e8-bb997de9854d None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:28.611495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ed7bd-04f6-448c-92e8-bb997de9854d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:28.611495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ed7bd-04f6-448c-92e8-bb997de9854d None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:28.612832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:28.618580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819a8b5-0d10-411d-a54f-214fc1a53dd3 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:28.619079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819a8b5-0d10-411d-a54f-214fc1a53dd3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:28.619079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819a8b5-0d10-411d-a54f-214fc1a53dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:28.704017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d819a8b5-0d10-411d-a54f-214fc1a53dd3 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:28.705250 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d819a8b5-0d10-411d-a54f-214fc1a53dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:28.768426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:28.818712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:28.865404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ed7bd-04f6-448c-92e8-bb997de9854d None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:28.867075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-272ed7bd-04f6-448c-92e8-bb997de9854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:29.030810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ed7bd-04f6-448c-92e8-bb997de9854d None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:29.116463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:29.158601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5ecf0-cc77-44cf-8089-26ec68757214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:29.159403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5ecf0-cc77-44cf-8089-26ec68757214 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:29.159790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5ecf0-cc77-44cf-8089-26ec68757214 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:29.160113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5ecf0-cc77-44cf-8089-26ec68757214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:29.162121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-bbc5ecf0-cc77-44cf-8089-26ec68757214 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:29.267675 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5ecf0-cc77-44cf-8089-26ec68757214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:29.269257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5ecf0-cc77-44cf-8089-26ec68757214 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:29.287716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 706/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:30.234720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5edea1-08b5-402f-849b-6f01ffd0b990 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:30.235306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5edea1-08b5-402f-849b-6f01ffd0b990 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:30.235306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5edea1-08b5-402f-849b-6f01ffd0b990 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:30.236682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:30.242462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e0def-604d-4038-b478-e37e33b4e306 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:30.242997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e0def-604d-4038-b478-e37e33b4e306 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:30.242997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e0def-604d-4038-b478-e37e33b4e306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:30.342322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-7b0e0def-604d-4038-b478-e37e33b4e306 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:30.343728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7b0e0def-604d-4038-b478-e37e33b4e306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:30.374017 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:30.445240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:30.603867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5edea1-08b5-402f-849b-6f01ffd0b990 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:30.605681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5d5edea1-08b5-402f-849b-6f01ffd0b990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:30.767674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5edea1-08b5-402f-849b-6f01ffd0b990 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:30.814082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:30.894387 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb3039-01fc-49d0-afe0-db079e40fea5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:30.894948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb3039-01fc-49d0-afe0-db079e40fea5 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:30.895240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb3039-01fc-49d0-afe0-db079e40fea5 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:30.895508 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb3039-01fc-49d0-afe0-db079e40fea5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:30.897340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2ceb3039-01fc-49d0-afe0-db079e40fea5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:31.001952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb3039-01fc-49d0-afe0-db079e40fea5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:31.003816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb3039-01fc-49d0-afe0-db079e40fea5 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:31.028750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 712/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:31.966888 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08aba43-b7fc-428f-9a82-098f6b69ca71 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:31.967569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08aba43-b7fc-428f-9a82-098f6b69ca71 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:31.967569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08aba43-b7fc-428f-9a82-098f6b69ca71 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:31.968816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:31.975206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117c212-17c5-4cc2-88ce-34f398c3e18f None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:31.975729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117c212-17c5-4cc2-88ce-34f398c3e18f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:31.975993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117c212-17c5-4cc2-88ce-34f398c3e18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:32.050987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:32.078201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-4117c212-17c5-4cc2-88ce-34f398c3e18f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:32.079431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4117c212-17c5-4cc2-88ce-34f398c3e18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:32.188385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:32.356173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08aba43-b7fc-428f-9a82-098f6b69ca71 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:32.359437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e08aba43-b7fc-428f-9a82-098f6b69ca71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:32.527757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08aba43-b7fc-428f-9a82-098f6b69ca71 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:32.542616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:32.650864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1126b-c699-48ec-8b25-542bd23ea2d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:32.651504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1126b-c699-48ec-8b25-542bd23ea2d3 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:32.651825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1126b-c699-48ec-8b25-542bd23ea2d3 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:32.652375 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1126b-c699-48ec-8b25-542bd23ea2d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:32.654206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-bef1126b-c699-48ec-8b25-542bd23ea2d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:32.759003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1126b-c699-48ec-8b25-542bd23ea2d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:32.760649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1126b-c699-48ec-8b25-542bd23ea2d3 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:32.782425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 709/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:33.693440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017c7e-74c1-4b0f-b8fa-9273984850f5 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:33.694698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017c7e-74c1-4b0f-b8fa-9273984850f5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:33.694698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017c7e-74c1-4b0f-b8fa-9273984850f5 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:33.696192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:33.702041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0469e5-2a35-445a-9ec4-8dea3b6297fc None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:33.702577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0469e5-2a35-445a-9ec4-8dea3b6297fc None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:33.702577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0469e5-2a35-445a-9ec4-8dea3b6297fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:33.723732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:33.792420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4d0469e5-2a35-445a-9ec4-8dea3b6297fc None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:33.793709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4d0469e5-2a35-445a-9ec4-8dea3b6297fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:33.909065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:34.072797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017c7e-74c1-4b0f-b8fa-9273984850f5 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:34.074896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5a017c7e-74c1-4b0f-b8fa-9273984850f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:34.188178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:34.226368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017c7e-74c1-4b0f-b8fa-9273984850f5 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:34.347832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d366177-1638-457c-9ae2-9efec6f534f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:34.348429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d366177-1638-457c-9ae2-9efec6f534f3 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:34.348699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d366177-1638-457c-9ae2-9efec6f534f3 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:34.348958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d366177-1638-457c-9ae2-9efec6f534f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:34.350680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8d366177-1638-457c-9ae2-9efec6f534f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:34.430251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:34.444851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d366177-1638-457c-9ae2-9efec6f534f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:34.446391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d366177-1638-457c-9ae2-9efec6f534f3 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:34.465521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 715/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:35.361188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc6187-0573-46aa-b314-27c376b46f4b None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:35.361705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc6187-0573-46aa-b314-27c376b46f4b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:35.361705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc6187-0573-46aa-b314-27c376b46f4b None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:35.363032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:35.368754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44489474-ed93-4b75-ab5f-8aeb989e08a7 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:35.369280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44489474-ed93-4b75-ab5f-8aeb989e08a7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:35.369374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44489474-ed93-4b75-ab5f-8aeb989e08a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:35.462050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-44489474-ed93-4b75-ab5f-8aeb989e08a7 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:35.463291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-44489474-ed93-4b75-ab5f-8aeb989e08a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:35.578651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:35.734177 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc6187-0573-46aa-b314-27c376b46f4b None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:35.735814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f1dc6187-0573-46aa-b314-27c376b46f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:35.835024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:35.912401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc6187-0573-46aa-b314-27c376b46f4b None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:36.052618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf614e-50a2-4aaf-b4f5-42f2fec4b871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:36.053152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf614e-50a2-4aaf-b4f5-42f2fec4b871 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:36.053435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf614e-50a2-4aaf-b4f5-42f2fec4b871 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:36.053870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf614e-50a2-4aaf-b4f5-42f2fec4b871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:36.055470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ccaf614e-50a2-4aaf-b4f5-42f2fec4b871 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:36.115708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:36.165353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf614e-50a2-4aaf-b4f5-42f2fec4b871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:36.167540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf614e-50a2-4aaf-b4f5-42f2fec4b871 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:36.190593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 712/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:37.139507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615eda8-92c4-42ed-a384-d023723ba6e3 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:37.140195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615eda8-92c4-42ed-a384-d023723ba6e3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:37.140195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615eda8-92c4-42ed-a384-d023723ba6e3 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:37.141619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:37.149389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dd40c-bdb5-4dd4-9602-199b166ef238 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:37.150010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dd40c-bdb5-4dd4-9602-199b166ef238 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:37.150010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dd40c-bdb5-4dd4-9602-199b166ef238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:37.388554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d86dd40c-bdb5-4dd4-9602-199b166ef238 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:37.390008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d86dd40c-bdb5-4dd4-9602-199b166ef238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:37.516888 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:37.671438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615eda8-92c4-42ed-a384-d023723ba6e3 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:37.673234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6615eda8-92c4-42ed-a384-d023723ba6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:37.731048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:37.864824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615eda8-92c4-42ed-a384-d023723ba6e3 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:38.001246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638b47f-5fea-4359-be2e-ef0255894810 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:38.002014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638b47f-5fea-4359-be2e-ef0255894810 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:38.002286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638b47f-5fea-4359-be2e-ef0255894810 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:38.002548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638b47f-5fea-4359-be2e-ef0255894810 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:38.004179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7638b47f-5fea-4359-be2e-ef0255894810 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:38.013010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:38.117559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638b47f-5fea-4359-be2e-ef0255894810 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:38.119268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638b47f-5fea-4359-be2e-ef0255894810 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:38.141655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 718/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:39.042013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926c787-d942-41fe-9c18-6309fcebca87 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:39.042927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926c787-d942-41fe-9c18-6309fcebca87 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:39.042927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926c787-d942-41fe-9c18-6309fcebca87 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:39.043970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:39.050256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7f698-d454-4e1f-91ed-9a21522ff8ae None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:39.050569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7f698-d454-4e1f-91ed-9a21522ff8ae None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:39.050842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7f698-d454-4e1f-91ed-9a21522ff8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:39.144664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f7e7f698-d454-4e1f-91ed-9a21522ff8ae None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:39.146076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f7e7f698-d454-4e1f-91ed-9a21522ff8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:39.222800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:39.254235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:39.409376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926c787-d942-41fe-9c18-6309fcebca87 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:39.410863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f926c787-d942-41fe-9c18-6309fcebca87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:39.431496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:39.591486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926c787-d942-41fe-9c18-6309fcebca87 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:39.716018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:39.749567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a2767-c030-49f6-8366-fd9d591cb9d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:39.750371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a2767-c030-49f6-8366-fd9d591cb9d8 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:39.750704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a2767-c030-49f6-8366-fd9d591cb9d8 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:39.751013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a2767-c030-49f6-8366-fd9d591cb9d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:39.752985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-be7a2767-c030-49f6-8366-fd9d591cb9d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:39.864142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a2767-c030-49f6-8366-fd9d591cb9d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:39.866289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a2767-c030-49f6-8366-fd9d591cb9d8 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:39.899577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 715/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:40.867658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96851531-8c58-4687-ad5b-07df5180caa1 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:40.868238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96851531-8c58-4687-ad5b-07df5180caa1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:40.868238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96851531-8c58-4687-ad5b-07df5180caa1 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:40.869537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:40.876247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9615419-39d2-4726-9fcd-36116efb7223 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:40.876775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9615419-39d2-4726-9fcd-36116efb7223 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:40.877149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9615419-39d2-4726-9fcd-36116efb7223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:40.960707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b9615419-39d2-4726-9fcd-36116efb7223 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:40.961932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b9615419-39d2-4726-9fcd-36116efb7223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:41.007577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:41.068283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:41.221469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96851531-8c58-4687-ad5b-07df5180caa1 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:41.223032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-96851531-8c58-4687-ad5b-07df5180caa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:41.376720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96851531-8c58-4687-ad5b-07df5180caa1 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:41.433547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:41.497161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd98e23-371a-426f-89db-9394910eab5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:41.497699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd98e23-371a-426f-89db-9394910eab5f service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:41.498004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd98e23-371a-426f-89db-9394910eab5f service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:41.498273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd98e23-371a-426f-89db-9394910eab5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:41.499940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4bd98e23-371a-426f-89db-9394910eab5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:41.598469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd98e23-371a-426f-89db-9394910eab5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:41.600073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd98e23-371a-426f-89db-9394910eab5f service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:41.619861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 721/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:42.682773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46b9b8-7fda-446c-81ef-3473e0ec383b None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:42.683326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46b9b8-7fda-446c-81ef-3473e0ec383b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:42.683326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46b9b8-7fda-446c-81ef-3473e0ec383b None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:42.684567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:53:42.690357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638d089-10af-4d07-93a3-54c652e82df8 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:42.690943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638d089-10af-4d07-93a3-54c652e82df8 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:42.690943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638d089-10af-4d07-93a3-54c652e82df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:42.776464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1638d089-10af-4d07-93a3-54c652e82df8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:42.777819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1638d089-10af-4d07-93a3-54c652e82df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:42.780185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:42.891545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:43.044603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46b9b8-7fda-446c-81ef-3473e0ec383b None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:43.046204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ef46b9b8-7fda-446c-81ef-3473e0ec383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:43.237014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46b9b8-7fda-446c-81ef-3473e0ec383b None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:43.269239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:43.408554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0861f56-5b49-425f-a626-f81219be7594 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:43.409419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0861f56-5b49-425f-a626-f81219be7594 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:43.409838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0861f56-5b49-425f-a626-f81219be7594 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:43.410118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0861f56-5b49-425f-a626-f81219be7594 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:43.413008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c0861f56-5b49-425f-a626-f81219be7594 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:43.555366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0861f56-5b49-425f-a626-f81219be7594 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:43.558728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0861f56-5b49-425f-a626-f81219be7594 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:43.592230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 718/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:45.556407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec68247-d6b9-4402-a550-03da7034e514 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.557475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec68247-d6b9-4402-a550-03da7034e514 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.557475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec68247-d6b9-4402-a550-03da7034e514 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:45.558558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:45.564204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6574b2d-46ac-4c35-9fab-a7cae4754dc3 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.564621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6574b2d-46ac-4c35-9fab-a7cae4754dc3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.564756 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6574b2d-46ac-4c35-9fab-a7cae4754dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:45.609126 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:45.637048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e6574b2d-46ac-4c35-9fab-a7cae4754dc3 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:45.638392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e6574b2d-46ac-4c35-9fab-a7cae4754dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:45.742938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:45.782069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec68247-d6b9-4402-a550-03da7034e514 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:45.783580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bec68247-d6b9-4402-a550-03da7034e514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:45.897989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec68247-d6b9-4402-a550-03da7034e514 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:45.935137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:45.980875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec47a3-654e-453c-9082-91b55cd800c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:45.981490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec47a3-654e-453c-9082-91b55cd800c9 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:45.981756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec47a3-654e-453c-9082-91b55cd800c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:45.982092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec47a3-654e-453c-9082-91b55cd800c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:45.983717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e1ec47a3-654e-453c-9082-91b55cd800c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:46.108691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec47a3-654e-453c-9082-91b55cd800c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:46.110091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec47a3-654e-453c-9082-91b55cd800c9 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:46.129888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 724/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:48.433541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568b411-214c-457c-bf75-0c9b4e818ce2 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:48.434142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568b411-214c-457c-bf75-0c9b4e818ce2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:48.434142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568b411-214c-457c-bf75-0c9b4e818ce2 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:48.435495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:48.441422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939cfec-0a51-4d9d-bef5-713450e2384f None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:48.441932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939cfec-0a51-4d9d-bef5-713450e2384f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:48.442029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939cfec-0a51-4d9d-bef5-713450e2384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:48.517260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2939cfec-0a51-4d9d-bef5-713450e2384f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:48.518524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2939cfec-0a51-4d9d-bef5-713450e2384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:48.533348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:48.626694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:48.665657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568b411-214c-457c-bf75-0c9b4e818ce2 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:48.668028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e568b411-214c-457c-bf75-0c9b4e818ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:48.781066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568b411-214c-457c-bf75-0c9b4e818ce2 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:48.794891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:48.868119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5978c-b548-4f7b-98b2-70441b4715da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:48.868748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5978c-b548-4f7b-98b2-70441b4715da service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:48.869016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5978c-b548-4f7b-98b2-70441b4715da service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:48.869281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5978c-b548-4f7b-98b2-70441b4715da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:48.870982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-68e5978c-b548-4f7b-98b2-70441b4715da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:49.011709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5978c-b548-4f7b-98b2-70441b4715da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:49.013221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5978c-b548-4f7b-98b2-70441b4715da service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:49.035197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 721/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:51.524245 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab9db5-cb8e-43b3-937d-7d88a01aaade None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.524808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab9db5-cb8e-43b3-937d-7d88a01aaade None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.524808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab9db5-cb8e-43b3-937d-7d88a01aaade None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.526855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:51.532338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7fdcb-810a-4b50-9e65-e812073a5d14 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.532850 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7fdcb-810a-4b50-9e65-e812073a5d14 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.532963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7fdcb-810a-4b50-9e65-e812073a5d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.583746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:51.603389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-91f7fdcb-810a-4b50-9e65-e812073a5d14 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:51.605349 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-91f7fdcb-810a-4b50-9e65-e812073a5d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:51.700657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:51.740751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab9db5-cb8e-43b3-937d-7d88a01aaade None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:51.742322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-03ab9db5-cb8e-43b3-937d-7d88a01aaade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:51.844891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab9db5-cb8e-43b3-937d-7d88a01aaade None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:51.883122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:51.917381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d327f3-c8ad-49be-bc6e-c073be95a9d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:51.918006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d327f3-c8ad-49be-bc6e-c073be95a9d8 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:51.918282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d327f3-c8ad-49be-bc6e-c073be95a9d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:51.918547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d327f3-c8ad-49be-bc6e-c073be95a9d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:51.920127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-28d327f3-c8ad-49be-bc6e-c073be95a9d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:52.029442 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d327f3-c8ad-49be-bc6e-c073be95a9d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:52.030939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d327f3-c8ad-49be-bc6e-c073be95a9d8 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:52.052166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 727/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:53:54.259834 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca0a6a-001e-4d1f-a3c2-659195b1103d None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:54.260391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca0a6a-001e-4d1f-a3c2-659195b1103d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:54.260391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca0a6a-001e-4d1f-a3c2-659195b1103d None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:54.261767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:54.267506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266042f-e60c-45a3-97d2-ac3ad6d719a8 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:54.268005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266042f-e60c-45a3-97d2-ac3ad6d719a8 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:54.268113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266042f-e60c-45a3-97d2-ac3ad6d719a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:54.339793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-8266042f-e60c-45a3-97d2-ac3ad6d719a8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:54.341009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8266042f-e60c-45a3-97d2-ac3ad6d719a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:54.363952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:54.444203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:54.484822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca0a6a-001e-4d1f-a3c2-659195b1103d None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:54.486404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5dca0a6a-001e-4d1f-a3c2-659195b1103d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:54.595199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca0a6a-001e-4d1f-a3c2-659195b1103d None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:54.615228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:54.667899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea62d6a-109e-49c8-b469-75bc1d20e198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:54.668495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea62d6a-109e-49c8-b469-75bc1d20e198 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:54.668769 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea62d6a-109e-49c8-b469-75bc1d20e198 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:54.669036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea62d6a-109e-49c8-b469-75bc1d20e198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:54.670696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9ea62d6a-109e-49c8-b469-75bc1d20e198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:54.776546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea62d6a-109e-49c8-b469-75bc1d20e198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:54.777979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea62d6a-109e-49c8-b469-75bc1d20e198 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:54.796472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 724/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:56.882766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf70334-8770-46dd-a5f9-383810dc4dd3 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:56.883351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf70334-8770-46dd-a5f9-383810dc4dd3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:56.883351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf70334-8770-46dd-a5f9-383810dc4dd3 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:56.884649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:56.890479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb4e20-e640-43fd-af82-7e970e2e47dd None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:56.891002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb4e20-e640-43fd-af82-7e970e2e47dd None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:56.891002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb4e20-e640-43fd-af82-7e970e2e47dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:56.959977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:56.962839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-00eb4e20-e640-43fd-af82-7e970e2e47dd None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:56.964510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-00eb4e20-e640-43fd-af82-7e970e2e47dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:57.063624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:53:57.103532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf70334-8770-46dd-a5f9-383810dc4dd3 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:57.105098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2bf70334-8770-46dd-a5f9-383810dc4dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:57.214407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf70334-8770-46dd-a5f9-383810dc4dd3 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:57.261633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:57.285978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb3cd5-0f91-49f5-a99f-b1dd8a299752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:53:57.286604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb3cd5-0f91-49f5-a99f-b1dd8a299752 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:57.286873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb3cd5-0f91-49f5-a99f-b1dd8a299752 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:57.287135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb3cd5-0f91-49f5-a99f-b1dd8a299752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:57.288703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-cadb3cd5-0f91-49f5-a99f-b1dd8a299752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:57.395196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb3cd5-0f91-49f5-a99f-b1dd8a299752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:53:57.396551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb3cd5-0f91-49f5-a99f-b1dd8a299752 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:53:57.415404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 730/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:53:59.585935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c2e93-c8e9-4f92-afaf-de6a33714d19 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:59.586494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c2e93-c8e9-4f92-afaf-de6a33714d19 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:59.586494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c2e93-c8e9-4f92-afaf-de6a33714d19 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:59.588074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:53:59.594030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024768e-7485-4ad2-b1c6-da3247e0d4a5 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:53:59.598223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024768e-7485-4ad2-b1c6-da3247e0d4a5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:53:59.598223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024768e-7485-4ad2-b1c6-da3247e0d4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:53:59.667226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9024768e-7485-4ad2-b1c6-da3247e0d4a5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:53:59.668470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9024768e-7485-4ad2-b1c6-da3247e0d4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:59.699303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:53:59.770403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:53:59.809399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c2e93-c8e9-4f92-afaf-de6a33714d19 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:53:59.810981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6d0c2e93-c8e9-4f92-afaf-de6a33714d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:53:59.924230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c2e93-c8e9-4f92-afaf-de6a33714d19 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:53:59.956421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:00.003793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf59523-a83d-4979-9eb6-43df78e768cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:00.004406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf59523-a83d-4979-9eb6-43df78e768cc service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:00.004682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf59523-a83d-4979-9eb6-43df78e768cc service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:00.004947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf59523-a83d-4979-9eb6-43df78e768cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:00.007098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ddf59523-a83d-4979-9eb6-43df78e768cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:00.125271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf59523-a83d-4979-9eb6-43df78e768cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:00.126678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf59523-a83d-4979-9eb6-43df78e768cc service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:00.151407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 727/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:53:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:02.290155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8f779-573a-48de-8bfe-cfbc70f12bac None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:02.290936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8f779-573a-48de-8bfe-cfbc70f12bac None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:02.290936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8f779-573a-48de-8bfe-cfbc70f12bac None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:02.292680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:54:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:02.299066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384478fb-b3aa-400c-8b1e-b04d12e2036f None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:02.299673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384478fb-b3aa-400c-8b1e-b04d12e2036f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:02.299785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384478fb-b3aa-400c-8b1e-b04d12e2036f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:02.371096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-384478fb-b3aa-400c-8b1e-b04d12e2036f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:02.372322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-384478fb-b3aa-400c-8b1e-b04d12e2036f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:02.381192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:02.477403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:02.516318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8f779-573a-48de-8bfe-cfbc70f12bac None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:02.517934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-23f8f779-573a-48de-8bfe-cfbc70f12bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:02.635795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8f779-573a-48de-8bfe-cfbc70f12bac None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:02.696488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:02.716884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36477b65-4484-40f1-9f58-7adfbe3dc67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:02.717620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36477b65-4484-40f1-9f58-7adfbe3dc67b service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:02.717930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36477b65-4484-40f1-9f58-7adfbe3dc67b service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:02.718203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36477b65-4484-40f1-9f58-7adfbe3dc67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:02.719826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-36477b65-4484-40f1-9f58-7adfbe3dc67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:02.840158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36477b65-4484-40f1-9f58-7adfbe3dc67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:02.841685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36477b65-4484-40f1-9f58-7adfbe3dc67b service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:02.861251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 733/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:04.962185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508b0f-e468-4cb1-a3e0-5e5e4574e333 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:04.963294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508b0f-e468-4cb1-a3e0-5e5e4574e333 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:04.963294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508b0f-e468-4cb1-a3e0-5e5e4574e333 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:04.964638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:04.970131 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d41bd4-a846-4eb9-9a5c-b06b98e0e7bf None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:04.970598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d41bd4-a846-4eb9-9a5c-b06b98e0e7bf None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:04.970715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d41bd4-a846-4eb9-9a5c-b06b98e0e7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:05.040166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-36d41bd4-a846-4eb9-9a5c-b06b98e0e7bf None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:05.041423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-36d41bd4-a846-4eb9-9a5c-b06b98e0e7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:05.081533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:05.141194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:05.178398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508b0f-e468-4cb1-a3e0-5e5e4574e333 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:05.179994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-81508b0f-e468-4cb1-a3e0-5e5e4574e333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:05.284258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508b0f-e468-4cb1-a3e0-5e5e4574e333 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:05.323213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:05.359285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479c5ff-0ca1-499c-a3dd-4e30905bfab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:05.359893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479c5ff-0ca1-499c-a3dd-4e30905bfab9 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:05.360172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479c5ff-0ca1-499c-a3dd-4e30905bfab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:05.360439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479c5ff-0ca1-499c-a3dd-4e30905bfab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:05.362076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e479c5ff-0ca1-499c-a3dd-4e30905bfab9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:05.467856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479c5ff-0ca1-499c-a3dd-4e30905bfab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:05.469296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479c5ff-0ca1-499c-a3dd-4e30905bfab9 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:05.488222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 730/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:07.582561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cc5b1-4e09-4873-8d47-baac3bf516c1 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.583241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cc5b1-4e09-4873-8d47-baac3bf516c1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.583241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cc5b1-4e09-4873-8d47-baac3bf516c1 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.584478 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:07.590296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba3a0c-6566-4a23-bfaa-0bf55dcd79a8 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.590711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba3a0c-6566-4a23-bfaa-0bf55dcd79a8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.590848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba3a0c-6566-4a23-bfaa-0bf55dcd79a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.656178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-0fba3a0c-6566-4a23-bfaa-0bf55dcd79a8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:07.657384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0fba3a0c-6566-4a23-bfaa-0bf55dcd79a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:07.675149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:07.754678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:07.791384 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cc5b1-4e09-4873-8d47-baac3bf516c1 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:07.792902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8f8cc5b1-4e09-4873-8d47-baac3bf516c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:07.895324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cc5b1-4e09-4873-8d47-baac3bf516c1 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:07.961135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:07.968270 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd2c7e-a0bc-4a74-88af-9e00b37d5cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:07.968875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd2c7e-a0bc-4a74-88af-9e00b37d5cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:07.969153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd2c7e-a0bc-4a74-88af-9e00b37d5cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:07.969424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd2c7e-a0bc-4a74-88af-9e00b37d5cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:07.971057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-91cd2c7e-a0bc-4a74-88af-9e00b37d5cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:08.079432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd2c7e-a0bc-4a74-88af-9e00b37d5cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:08.080896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd2c7e-a0bc-4a74-88af-9e00b37d5cc2 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:08.100564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 736/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:10.192939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e9734-3bfd-41f8-83ed-8e12af0295b0 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:10.193761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e9734-3bfd-41f8-83ed-8e12af0295b0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:10.193761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e9734-3bfd-41f8-83ed-8e12af0295b0 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:10.196521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:54:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:10.202481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf5b02-b565-425c-b32a-d8d6ac1bb582 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:10.202980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf5b02-b565-425c-b32a-d8d6ac1bb582 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:10.203102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf5b02-b565-425c-b32a-d8d6ac1bb582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:10.271761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f9cf5b02-b565-425c-b32a-d8d6ac1bb582 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:10.272998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f9cf5b02-b565-425c-b32a-d8d6ac1bb582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:10.326603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:10.382147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:10.423052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e9734-3bfd-41f8-83ed-8e12af0295b0 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:10.424590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f38e9734-3bfd-41f8-83ed-8e12af0295b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:10.552329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e9734-3bfd-41f8-83ed-8e12af0295b0 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:10.613420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:10.640720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182d236-8cb7-4078-b108-f01f017028f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:10.641407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182d236-8cb7-4078-b108-f01f017028f5 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:10.641722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182d236-8cb7-4078-b108-f01f017028f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:10.642022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182d236-8cb7-4078-b108-f01f017028f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:10.643626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9182d236-8cb7-4078-b108-f01f017028f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:10.765574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182d236-8cb7-4078-b108-f01f017028f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:10.767139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182d236-8cb7-4078-b108-f01f017028f5 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:10.787117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 733/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:20.379464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=22 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:20.805727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=66 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:31.533938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243283b8-647f-475a-9a23-54ccba52cc3d None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:31.534520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243283b8-647f-475a-9a23-54ccba52cc3d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:31.534639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243283b8-647f-475a-9a23-54ccba52cc3d None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:31.535986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 738/1471] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:54:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:31.541905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713ff8c-30fe-4ed4-8559-d60eb89cb922 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:31.546135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713ff8c-30fe-4ed4-8559-d60eb89cb922 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:31.546135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713ff8c-30fe-4ed4-8559-d60eb89cb922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:31.622025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-7713ff8c-30fe-4ed4-8559-d60eb89cb922 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:31.623282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7713ff8c-30fe-4ed4-8559-d60eb89cb922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:31.723512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 734/1472] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:31.823615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf88d8-8ee5-4331-8af5-8bad4164b1c7 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:31.823894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf88d8-8ee5-4331-8af5-8bad4164b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:31.824169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf88d8-8ee5-4331-8af5-8bad4164b1c7 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:31.825628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 739/1473] 10.222.0.57 () {54 vars in 967 bytes} [Mon Aug 8 16:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:31.831309 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4da7d-4cdf-4ca3-aae1-2990e947cdc2 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:31.831597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4da7d-4cdf-4ca3-aae1-2990e947cdc2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:31.831874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4da7d-4cdf-4ca3-aae1-2990e947cdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:31.927833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e0c4da7d-4cdf-4ca3-aae1-2990e947cdc2 None None] MFA Rules not processed for user `73bfe655c6d34918b21acb8748b9156d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:31.929055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e0c4da7d-4cdf-4ca3-aae1-2990e947cdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:31.951845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:32.026070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 735/1474] 10.222.0.57 () {58 vars in 1064 bytes} [Mon Aug 8 16:54:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:32.034175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf88d8-8ee5-4331-8af5-8bad4164b1c7 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:32.036262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bcbf88d8-8ee5-4331-8af5-8bad4164b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:32.161858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf88d8-8ee5-4331-8af5-8bad4164b1c7 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:32.246837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782acbc-cbb1-449a-a1ee-e89b3dd0e029 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73bfe655c6d34918b21acb8748b9156d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:32.247495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782acbc-cbb1-449a-a1ee-e89b3dd0e029 service glance] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:32.247843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782acbc-cbb1-449a-a1ee-e89b3dd0e029 service glance] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:32.248154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782acbc-cbb1-449a-a1ee-e89b3dd0e029 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:32.250095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f782acbc-cbb1-449a-a1ee-e89b3dd0e029 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:32.329333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:32.395441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782acbc-cbb1-449a-a1ee-e89b3dd0e029 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:32.396924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782acbc-cbb1-449a-a1ee-e89b3dd0e029 service glance] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:32.420942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 740/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:34.584124 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8905a7-f1f6-400c-a6f5-901baea3a900 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:34.584961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8905a7-f1f6-400c-a6f5-901baea3a900 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:34.584961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8905a7-f1f6-400c-a6f5-901baea3a900 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:34.587322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 736/1476] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:54:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:54:34.596151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afdd2a-0965-47cc-a8d2-81eb1c1ed9d9 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:34.596700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afdd2a-0965-47cc-a8d2-81eb1c1ed9d9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:34.597032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afdd2a-0965-47cc-a8d2-81eb1c1ed9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:34.689844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-10afdd2a-0965-47cc-a8d2-81eb1c1ed9d9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:34.691052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-10afdd2a-0965-47cc-a8d2-81eb1c1ed9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:34.860118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 741/1477] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:34.890225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8905a7-f1f6-400c-a6f5-901baea3a900 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:34.891838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ac8905a7-f1f6-400c-a6f5-901baea3a900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:35.029415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8905a7-f1f6-400c-a6f5-901baea3a900 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:35.091970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:35.151388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932a7cd-9e40-4c95-8468-6cb94fbb5e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:35.151891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a7cd-9e40-4c95-8468-6cb94fbb5e99 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:35.152157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a7cd-9e40-4c95-8468-6cb94fbb5e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:35.152431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a7cd-9e40-4c95-8468-6cb94fbb5e99 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:35.153069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932a7cd-9e40-4c95-8468-6cb94fbb5e99 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:35.154855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING py.warnings [None req-3932a7cd-9e40-4c95-8468-6cb94fbb5e99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:54:35.154855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: warnings.warn(msg) Aug 08 16:54:35.154855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: Aug 08 16:54:35.156007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932a7cd-9e40-4c95-8468-6cb94fbb5e99 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:35.179956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 737/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Aug 8 16:54:34 2022] GET /identity/v3/projects => generated 2553 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:54:37.299954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770c6cf-0845-417f-8ade-3668c7565553 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:37.300835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770c6cf-0845-417f-8ade-3668c7565553 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:37.301035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770c6cf-0845-417f-8ade-3668c7565553 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:37.303372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 742/1479] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:54:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:37.309208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a2539-9d00-46cf-a929-72723fff44e9 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:37.309722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a2539-9d00-46cf-a929-72723fff44e9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:37.310331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a2539-9d00-46cf-a929-72723fff44e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:37.391411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c37a2539-9d00-46cf-a929-72723fff44e9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:37.392651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c37a2539-9d00-46cf-a929-72723fff44e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:37.505562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 738/1480] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:37.556604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770c6cf-0845-417f-8ade-3668c7565553 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:37.558185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6770c6cf-0845-417f-8ade-3668c7565553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:37.569406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:37.737760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770c6cf-0845-417f-8ade-3668c7565553 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:37.841088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:37.875449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c1a66-f408-4d6d-b0ea-fa84e57d2c02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:37.876132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c1a66-f408-4d6d-b0ea-fa84e57d2c02 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:37.876475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c1a66-f408-4d6d-b0ea-fa84e57d2c02 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:37.876787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c1a66-f408-4d6d-b0ea-fa84e57d2c02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:37.879383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3a7c1a66-f408-4d6d-b0ea-fa84e57d2c02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:37.994485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c1a66-f408-4d6d-b0ea-fa84e57d2c02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:37.996081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c1a66-f408-4d6d-b0ea-fa84e57d2c02 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:38.025704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 743/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:40.290303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9397e1-bbbd-4728-99db-43b0147089ba None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:40.290949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9397e1-bbbd-4728-99db-43b0147089ba None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:40.291268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9397e1-bbbd-4728-99db-43b0147089ba None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:40.293724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 739/1482] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:54:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:40.299393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e89979-04b9-463b-8167-e001f8caedba None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:40.300139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e89979-04b9-463b-8167-e001f8caedba None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:40.300139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e89979-04b9-463b-8167-e001f8caedba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:40.371823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-82e89979-04b9-463b-8167-e001f8caedba None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:40.373048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-82e89979-04b9-463b-8167-e001f8caedba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:40.426619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:40.495231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 744/1483] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:54:40.533454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9397e1-bbbd-4728-99db-43b0147089ba None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:40.535048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6a9397e1-bbbd-4728-99db-43b0147089ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:40.574511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:40.685305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9397e1-bbbd-4728-99db-43b0147089ba None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:40.765293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ff0e2-8c29-4683-9754-30b3fb108c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:40.765945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ff0e2-8c29-4683-9754-30b3fb108c40 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:40.766229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ff0e2-8c29-4683-9754-30b3fb108c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:40.766469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ff0e2-8c29-4683-9754-30b3fb108c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:40.768122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-449ff0e2-8c29-4683-9754-30b3fb108c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:40.870427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:40.925898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ff0e2-8c29-4683-9754-30b3fb108c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:40.927331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ff0e2-8c29-4683-9754-30b3fb108c40 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:40.946690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 740/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:54:43.093715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0224f43-377f-4cfc-8c1d-19bcce31fe91 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:43.094399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0224f43-377f-4cfc-8c1d-19bcce31fe91 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:43.094634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0224f43-377f-4cfc-8c1d-19bcce31fe91 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:43.096954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 745/1485] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:54:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:43.104337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109afbc4-09c1-4cd1-a749-86e1b4138604 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:43.105235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109afbc4-09c1-4cd1-a749-86e1b4138604 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:43.105235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109afbc4-09c1-4cd1-a749-86e1b4138604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:43.187582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-109afbc4-09c1-4cd1-a749-86e1b4138604 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:43.188832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-109afbc4-09c1-4cd1-a749-86e1b4138604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:43.307567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 741/1486] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:43.371044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0224f43-377f-4cfc-8c1d-19bcce31fe91 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:43.372596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b0224f43-377f-4cfc-8c1d-19bcce31fe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:43.484859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0224f43-377f-4cfc-8c1d-19bcce31fe91 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:43.587967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:43.600630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada682a0-a444-449d-a255-5598013b96a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:43.600630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada682a0-a444-449d-a255-5598013b96a7 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:43.600630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada682a0-a444-449d-a255-5598013b96a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:43.600630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada682a0-a444-449d-a255-5598013b96a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:43.601068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ada682a0-a444-449d-a255-5598013b96a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:43.736282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada682a0-a444-449d-a255-5598013b96a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:43.737716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada682a0-a444-449d-a255-5598013b96a7 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:43.758115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 746/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:46.018261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72518fb2-770f-46f5-b033-dee07cc44066 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:46.019135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72518fb2-770f-46f5-b033-dee07cc44066 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:46.019135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72518fb2-770f-46f5-b033-dee07cc44066 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:46.020854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 742/1488] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:54:46.029063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3434a2-2acc-4981-8d8e-935211823f6e None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:46.029954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3434a2-2acc-4981-8d8e-935211823f6e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:46.029954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3434a2-2acc-4981-8d8e-935211823f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:46.112516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0c3434a2-2acc-4981-8d8e-935211823f6e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:46.113737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0c3434a2-2acc-4981-8d8e-935211823f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:46.175677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:46.232670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 747/1489] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:46.282886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72518fb2-770f-46f5-b033-dee07cc44066 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:46.284748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-72518fb2-770f-46f5-b033-dee07cc44066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:46.328983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:46.420051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72518fb2-770f-46f5-b033-dee07cc44066 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:46.507051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e729dd5-40c0-4dcb-b01b-ccec0d7f3393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:46.508082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e729dd5-40c0-4dcb-b01b-ccec0d7f3393 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:46.508365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e729dd5-40c0-4dcb-b01b-ccec0d7f3393 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:46.508639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e729dd5-40c0-4dcb-b01b-ccec0d7f3393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:46.510425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4e729dd5-40c0-4dcb-b01b-ccec0d7f3393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:46.631547 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:46.665204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e729dd5-40c0-4dcb-b01b-ccec0d7f3393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:46.666869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e729dd5-40c0-4dcb-b01b-ccec0d7f3393 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:46.687976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 743/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:49.305676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de24479-dd90-4a1d-8415-20be88f4a9ae None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:49.306355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de24479-dd90-4a1d-8415-20be88f4a9ae None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:49.306355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de24479-dd90-4a1d-8415-20be88f4a9ae None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:49.307593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 748/1491] 10.222.0.57 () {54 vars in 927 bytes} [Mon Aug 8 16:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:54:49.313229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38e8c-6cad-4340-97d8-31c9fe4a9012 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:49.313676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38e8c-6cad-4340-97d8-31c9fe4a9012 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:49.313813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38e8c-6cad-4340-97d8-31c9fe4a9012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:49.383403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a6a38e8c-6cad-4340-97d8-31c9fe4a9012 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:54:49.384654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a6a38e8c-6cad-4340-97d8-31c9fe4a9012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:49.512351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 744/1492] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Aug 8 16:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:54:49.553748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de24479-dd90-4a1d-8415-20be88f4a9ae None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:54:49.555562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9de24479-dd90-4a1d-8415-20be88f4a9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:49.718904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de24479-dd90-4a1d-8415-20be88f4a9ae None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:54:49.826457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80708b-4d72-42c8-a4f2-9f54945dd75b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:54:49.827150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80708b-4d72-42c8-a4f2-9f54945dd75b service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:54:49.827431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80708b-4d72-42c8-a4f2-9f54945dd75b service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:54:49.827700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80708b-4d72-42c8-a4f2-9f54945dd75b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:54:49.829363 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4c80708b-4d72-42c8-a4f2-9f54945dd75b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:54:49.841316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:49.965421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80708b-4d72-42c8-a4f2-9f54945dd75b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:54:49.966937 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80708b-4d72-42c8-a4f2-9f54945dd75b service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:54:49.987732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 749/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:54:59.510020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=82 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:54:59.985085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=50 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:16.025822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2aa55-cf18-4c7f-8497-1bd7f82f43dd None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:16.026423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2aa55-cf18-4c7f-8497-1bd7f82f43dd None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:16.026423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2aa55-cf18-4c7f-8497-1bd7f82f43dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:16.106703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4ee2aa55-cf18-4c7f-8497-1bd7f82f43dd None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:56:16.107934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4ee2aa55-cf18-4c7f-8497-1bd7f82f43dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:16.206687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 745/1494] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 16:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:56:16.216805 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0a3a4-6c4c-4312-bc4d-9f60f188e318 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:16.217328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0a3a4-6c4c-4312-bc4d-9f60f188e318 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:16.217328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0a3a4-6c4c-4312-bc4d-9f60f188e318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:16.295658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-50c0a3a4-6c4c-4312-bc4d-9f60f188e318 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:56:16.296868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-50c0a3a4-6c4c-4312-bc4d-9f60f188e318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:16.393428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 750/1495] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 16:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:56:16.400747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2aa55-cf18-4c7f-8497-1bd7f82f43dd None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:16.402153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4ee2aa55-cf18-4c7f-8497-1bd7f82f43dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:16.414150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2aa55-cf18-4c7f-8497-1bd7f82f43dd None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:16.480042 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:16.516392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:16.517132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:16.517412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:16.517680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:16.518489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 16:56:16.518791 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:16.519796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING py.warnings [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:56:16.519796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: warnings.warn(msg) Aug 08 16:56:16.519796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: Aug 08 16:56:16.520747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:16.532269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 746/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 16:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:56:16.540967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0a3a4-6c4c-4312-bc4d-9f60f188e318 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:16.550391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0a3a4-6c4c-4312-bc4d-9f60f188e318 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:16.611827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:16.647652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:16.648729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:16.651875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:16.652154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:16.653071 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:16.655954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: WARNING py.warnings [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:56:16.655954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: warnings.warn(msg) Aug 08 16:56:16.655954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: Aug 08 16:56:16.657148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:16.786139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 751/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 16:56:16 2022] POST /identity/v3/projects => generated 379 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:56:16.790531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:16.792058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:16.827317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27f7a2-fdee-4f7f-a00b-246aa6c7b5f2 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:16.931794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:16.932396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:16.932664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:16.932923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:16.933531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:16.934566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING py.warnings [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:56:16.934566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: warnings.warn(msg) Aug 08 16:56:16.934566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: Aug 08 16:56:16.935761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:17.078045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 747/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 16:56:16 2022] POST /identity/v3/users => generated 290 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 16:56:17.082228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:17.083772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:17.150717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738c21d-8f7e-4785-a086-87f69cfb5efd admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:17.250134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:17.261894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:17.263856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:17.263856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:17.263856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:17.264998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:17.266542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: WARNING py.warnings [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:56:17.266542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: warnings.warn(msg) Aug 08 16:56:17.266542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: Aug 08 16:56:17.267441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:17.282007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 752/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 16:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:56:17.286940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:17.289414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:17.301261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b4157-3c93-49c9-a687-475356364788 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:17.336315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:17.422623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555bae6-73f4-4078-ad13-97a873c933f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:17.423234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555bae6-73f4-4078-ad13-97a873c933f6 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:17.423510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555bae6-73f4-4078-ad13-97a873c933f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:17.423770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555bae6-73f4-4078-ad13-97a873c933f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:17.424416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555bae6-73f4-4078-ad13-97a873c933f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:17.425469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING py.warnings [None req-b555bae6-73f4-4078-ad13-97a873c933f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:56:17.425469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: warnings.warn(msg) Aug 08 16:56:17.425469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: Aug 08 16:56:17.426405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555bae6-73f4-4078-ad13-97a873c933f6 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:17.440825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 748/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 16:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 16:56:17.446939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:17.449401 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:17.485061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779e2ae-dc52-4059-8028-4d02e0cb61f0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:17.589441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe246e97-608c-469a-a69e-ab97604e80b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:17.590135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe246e97-608c-469a-a69e-ab97604e80b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:17.590408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe246e97-608c-469a-a69e-ab97604e80b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:17.590672 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe246e97-608c-469a-a69e-ab97604e80b2 admin admin] PATH_INFO: `/v3/projects/8210f0b9b1294b76be39e298262ce747/users/7f8cf68dee374f1982d6ee7f2883f14f/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:17.614723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe246e97-608c-469a-a69e-ab97604e80b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8210f0b9b1294b76be39e298262ce747, user_id=7f8cf68dee374f1982d6ee7f2883f14f, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:17.615753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: WARNING py.warnings [None req-fe246e97-608c-469a-a69e-ab97604e80b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:56:17.615753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: warnings.warn(msg) Aug 08 16:56:17.615753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: Aug 08 16:56:17.617755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe246e97-608c-469a-a69e-ab97604e80b2 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:17.730618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 753/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 16:56:17 2022] PUT /identity/v3/projects/8210f0b9b1294b76be39e298262ce747/users/7f8cf68dee374f1982d6ee7f2883f14f/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:56:17.743898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0056a-2351-4d06-b4ab-03d467e25a5e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:17.747167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0056a-2351-4d06-b4ab-03d467e25a5e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:17.747167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0056a-2351-4d06-b4ab-03d467e25a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:17.805528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f7a0056a-2351-4d06-b4ab-03d467e25a5e None None] MFA Rules not processed for user `7f8cf68dee374f1982d6ee7f2883f14f`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:56:17.807258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f7a0056a-2351-4d06-b4ab-03d467e25a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:17.887070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 749/1502] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 16:56:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:56:17.902566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5917789-52f1-43f7-b661-820683c90b5e None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:17.903048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5917789-52f1-43f7-b661-820683c90b5e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:17.903499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5917789-52f1-43f7-b661-820683c90b5e None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:17.905862 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 754/1503] 10.222.0.57 () {54 vars in 964 bytes} [Mon Aug 8 16:56:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:56:17.911357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e401f9a-1cca-4849-822d-4501f715d205 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:17.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e401f9a-1cca-4849-822d-4501f715d205 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:17.912053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e401f9a-1cca-4849-822d-4501f715d205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:17.950452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:18.010955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6e401f9a-1cca-4849-822d-4501f715d205 None None] MFA Rules not processed for user `e860aad24688491c9a89ddfb35922eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:56:18.012157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6e401f9a-1cca-4849-822d-4501f715d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:18.095190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 750/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Aug 8 16:56:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:56:18.103305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5917789-52f1-43f7-b661-820683c90b5e None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:18.105132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f5917789-52f1-43f7-b661-820683c90b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:18.157200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:18.214250 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5917789-52f1-43f7-b661-820683c90b5e None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:18.296391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:18.297392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:18.297392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:18.297591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:18.299132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:18.410769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:18.412209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:18.417347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:18.432355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 755/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:56:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:56:18.480577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e17b1-2862-4294-9e47-c21e85d86907 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:18.480860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e17b1-2862-4294-9e47-c21e85d86907 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:18.481130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e17b1-2862-4294-9e47-c21e85d86907 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:18.482422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 751/1506] 10.222.0.57 () {52 vars in 830 bytes} [Mon Aug 8 16:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:56:18.486224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:18.487466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:18.597523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69168497-5815-403b-93a2-c34b7aa0ae05 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:18.693123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:18.693694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:18.693694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:18.695677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] PATH_INFO: `/v3/users/7f8cf68dee374f1982d6ee7f2883f14f` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:18.707312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8cf68dee374f1982d6ee7f2883f14f)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:18.708336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: WARNING py.warnings [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:56:18.708336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: warnings.warn(msg) Aug 08 16:56:18.708336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: Aug 08 16:56:18.709620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:18.721677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:18.950012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8cf68dee374f1982d6ee7f2883f14f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:19.047007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8cf68dee374f1982d6ee7f2883f14f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:19.073136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8cf68dee374f1982d6ee7f2883f14f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:19.087037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8cf68dee374f1982d6ee7f2883f14f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:19.087615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8cf68dee374f1982d6ee7f2883f14f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:19.093297 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 756/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 16:56:18 2022] DELETE /identity/v3/users/7f8cf68dee374f1982d6ee7f2883f14f => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:56:19.097892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fa14d-3167-46b3-beb2-9ee63a0f75ef None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:19.098537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fa14d-3167-46b3-beb2-9ee63a0f75ef None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:19.098537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fa14d-3167-46b3-beb2-9ee63a0f75ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:19.159964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-0d4fa14d-3167-46b3-beb2-9ee63a0f75ef None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:56:19.162172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0d4fa14d-3167-46b3-beb2-9ee63a0f75ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:19.238746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:19.251809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 752/1508] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 16:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:56:19.264022 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:19.266234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:19.391427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e5135-7acb-4822-8bda-44fc93633d4d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:19.454783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:19.475527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ebd20-d358-4f48-b21a-37ad0ff94d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:19.476712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ebd20-d358-4f48-b21a-37ad0ff94d11 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:19.476987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ebd20-d358-4f48-b21a-37ad0ff94d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:19.477254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ebd20-d358-4f48-b21a-37ad0ff94d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:19.478953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c73ebd20-d358-4f48-b21a-37ad0ff94d11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:19.612779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ebd20-d358-4f48-b21a-37ad0ff94d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:19.614262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ebd20-d358-4f48-b21a-37ad0ff94d11 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:19.633138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 757/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 16:56:20.182380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fa14d-3167-46b3-beb2-9ee63a0f75ef None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:56:20.183792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0d4fa14d-3167-46b3-beb2-9ee63a0f75ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:56:20.202945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fa14d-3167-46b3-beb2-9ee63a0f75ef None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:56:20.352070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:56:20.352685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:56:20.352989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:56:20.353254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] PATH_INFO: `/v3/projects/8210f0b9b1294b76be39e298262ce747` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:56:20.364102 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8210f0b9b1294b76be39e298262ce747)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:56:20.365141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING py.warnings [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 16:56:20.365141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: warnings.warn(msg) Aug 08 16:56:20.365141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: Aug 08 16:56:20.366389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:56:20.497870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8210f0b9b1294b76be39e298262ce747', 'request_id': 'req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:20.515958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8210f0b9b1294b76be39e298262ce747', 'request_id': 'req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:20.641606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8210f0b9b1294b76be39e298262ce747', 'request_id': 'req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:20.667522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] The token cache is being invalidate because project 8210f0b9b1294b76be39e298262ce747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 16:56:20.667838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:20.668449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4f3af90b-8b6f-4174-b2bb-da022d9eaecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 16:56:20.669811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 753/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 16:56:20 2022] DELETE /identity/v3/projects/8210f0b9b1294b76be39e298262ce747 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 16:56:29.630702 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=62 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:56:30.665257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:57:51.354936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f16ef-f0ab-4b0f-8816-8dc361d29edd None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:51.355783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f16ef-f0ab-4b0f-8816-8dc361d29edd None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:51.355783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f16ef-f0ab-4b0f-8816-8dc361d29edd None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:51.357348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:57:51.364825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedfd78-f8c4-4d5e-b2e7-f4cfaeeafb4e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:51.365646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedfd78-f8c4-4d5e-b2e7-f4cfaeeafb4e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:51.365646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedfd78-f8c4-4d5e-b2e7-f4cfaeeafb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:51.457513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6cedfd78-f8c4-4d5e-b2e7-f4cfaeeafb4e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:57:51.458784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6cedfd78-f8c4-4d5e-b2e7-f4cfaeeafb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:51.568907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:57:51.734133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f16ef-f0ab-4b0f-8816-8dc361d29edd None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:51.741296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-963f16ef-f0ab-4b0f-8816-8dc361d29edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:51.865049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f16ef-f0ab-4b0f-8816-8dc361d29edd None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:51.871104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073dd07-ab2d-4421-928e-d62bbb5aeb73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:51.871705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073dd07-ab2d-4421-928e-d62bbb5aeb73 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:51.871984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073dd07-ab2d-4421-928e-d62bbb5aeb73 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:51.872254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073dd07-ab2d-4421-928e-d62bbb5aeb73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:51.880706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073dd07-ab2d-4421-928e-d62bbb5aeb73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:51.882376 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073dd07-ab2d-4421-928e-d62bbb5aeb73 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:51.893134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 759/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:52.770208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1190956-e89e-437c-8a70-95bd096def70 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:52.770776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1190956-e89e-437c-8a70-95bd096def70 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:52.770776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1190956-e89e-437c-8a70-95bd096def70 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:52.772087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:57:52.777913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c7fcc-cbf3-4f60-8a1e-0e0a8f713f57 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:52.778389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c7fcc-cbf3-4f60-8a1e-0e0a8f713f57 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:52.778538 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c7fcc-cbf3-4f60-8a1e-0e0a8f713f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:52.801665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-cd6c7fcc-cbf3-4f60-8a1e-0e0a8f713f57 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:57:52.803180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-cd6c7fcc-cbf3-4f60-8a1e-0e0a8f713f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:52.811204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:57:52.966329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1190956-e89e-437c-8a70-95bd096def70 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:57:52.986698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1190956-e89e-437c-8a70-95bd096def70 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:57:52.992382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ca7a5-a7c5-4ce9-ae60-3d4691089a53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:57:52.993022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ca7a5-a7c5-4ce9-ae60-3d4691089a53 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:52.993293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ca7a5-a7c5-4ce9-ae60-3d4691089a53 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:52.993669 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ca7a5-a7c5-4ce9-ae60-3d4691089a53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:53.002351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ca7a5-a7c5-4ce9-ae60-3d4691089a53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:57:53.004224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ca7a5-a7c5-4ce9-ae60-3d4691089a53 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:57:53.015531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 756/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:57:55.691562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d5fa-cf83-4fdf-8aa1-a34f1558817c None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:55.692133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d5fa-cf83-4fdf-8aa1-a34f1558817c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:55.692133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d5fa-cf83-4fdf-8aa1-a34f1558817c None None] PATH_INFO: `/v3` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:55.693385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 761/1517] 10.222.0.67 () {54 vars in 925 bytes} [Mon Aug 8 16:57:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 16:57:55.700977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c9c4d-0c1b-4fde-935a-9af2dd4886d3 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:57:55.701432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c9c4d-0c1b-4fde-935a-9af2dd4886d3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:57:55.701546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c9c4d-0c1b-4fde-935a-9af2dd4886d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:57:55.765714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-9f1c9c4d-0c1b-4fde-935a-9af2dd4886d3 None None] MFA Rules not processed for user `5853f9b60cf34380b218a5cda94f587d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:57:55.766969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9f1c9c4d-0c1b-4fde-935a-9af2dd4886d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:57:55.784762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:57:55.847557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 757/1518] 10.222.0.67 () {58 vars in 1015 bytes} [Mon Aug 8 16:57:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:58:02.797117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=56 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:04.273203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64920c84-d984-4bbc-a007-d002acb70fdc None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:04.273859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64920c84-d984-4bbc-a007-d002acb70fdc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:04.273859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64920c84-d984-4bbc-a007-d002acb70fdc None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:04.276104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 762/1519] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:58:04.281083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b6c5-a3bf-40a4-918e-d29c1988131d None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:04.281616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b6c5-a3bf-40a4-918e-d29c1988131d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:04.281616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b6c5-a3bf-40a4-918e-d29c1988131d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:04.302056 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b3b5b6c5-a3bf-40a4-918e-d29c1988131d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:04.303277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b3b5b6c5-a3bf-40a4-918e-d29c1988131d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:04.312177 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 758/1520] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 16:58:04.475723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64920c84-d984-4bbc-a007-d002acb70fdc None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:04.485305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64920c84-d984-4bbc-a007-d002acb70fdc None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:04.495214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb68a7-9487-4d6f-993c-ec705c6525a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:04.496166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb68a7-9487-4d6f-993c-ec705c6525a5 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:04.496616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb68a7-9487-4d6f-993c-ec705c6525a5 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:04.497059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb68a7-9487-4d6f-993c-ec705c6525a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:04.508230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb68a7-9487-4d6f-993c-ec705c6525a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:04.510534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb68a7-9487-4d6f-993c-ec705c6525a5 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:04.520798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 763/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:05.361945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a79edb-f9fc-493a-8d9e-fa358f3248a1 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:05.362545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a79edb-f9fc-493a-8d9e-fa358f3248a1 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:05.362654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a79edb-f9fc-493a-8d9e-fa358f3248a1 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:05.364077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:58:05.369385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ddc8e-157b-48e9-b55f-9197c8a7a5a1 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:05.369858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ddc8e-157b-48e9-b55f-9197c8a7a5a1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:05.369967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ddc8e-157b-48e9-b55f-9197c8a7a5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:05.389913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-158ddc8e-157b-48e9-b55f-9197c8a7a5a1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:05.391137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-158ddc8e-157b-48e9-b55f-9197c8a7a5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:05.400905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:05.561439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a79edb-f9fc-493a-8d9e-fa358f3248a1 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:05.576975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a79edb-f9fc-493a-8d9e-fa358f3248a1 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:05.583279 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bd1be-568b-4644-b808-ceb024fd9660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:05.583889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bd1be-568b-4644-b808-ceb024fd9660 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:05.584195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bd1be-568b-4644-b808-ceb024fd9660 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:05.584472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bd1be-568b-4644-b808-ceb024fd9660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:05.594719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bd1be-568b-4644-b808-ceb024fd9660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:05.596330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bd1be-568b-4644-b808-ceb024fd9660 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:05.607100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 760/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:07.474146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e889d67-8a6a-49e2-b84b-22165194aa3d None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:07.474912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e889d67-8a6a-49e2-b84b-22165194aa3d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:07.474912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e889d67-8a6a-49e2-b84b-22165194aa3d None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:07.476134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 765/1525] 10.222.0.57 () {54 vars in 961 bytes} [Mon Aug 8 16:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:58:07.482134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f10d7-7ef5-4929-a456-a05fc2994630 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:07.482917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f10d7-7ef5-4929-a456-a05fc2994630 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:07.482917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f10d7-7ef5-4929-a456-a05fc2994630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:07.504612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-115f10d7-7ef5-4929-a456-a05fc2994630 None None] MFA Rules not processed for user `5853f9b60cf34380b218a5cda94f587d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:07.505943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-115f10d7-7ef5-4929-a456-a05fc2994630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:07.515213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 761/1526] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Aug 8 16:58:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:07.522947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e889d67-8a6a-49e2-b84b-22165194aa3d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:07.537914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e889d67-8a6a-49e2-b84b-22165194aa3d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:07.543751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfc991-e6d9-46e7-80e6-b0056153a081 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:07.544584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfc991-e6d9-46e7-80e6-b0056153a081 service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:07.544858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfc991-e6d9-46e7-80e6-b0056153a081 service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:07.545126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfc991-e6d9-46e7-80e6-b0056153a081 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:07.556120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfc991-e6d9-46e7-80e6-b0056153a081 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:07.557712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfc991-e6d9-46e7-80e6-b0056153a081 service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:07.567735 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 766/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:16.857210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46f1fe-fa93-4338-bed8-6e7f80e9ba39 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:16.857838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46f1fe-fa93-4338-bed8-6e7f80e9ba39 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:16.857838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46f1fe-fa93-4338-bed8-6e7f80e9ba39 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:16.859096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 16:58:16.864325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e82c-e20f-429f-8da2-8e6faec3eef7 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:16.864870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e82c-e20f-429f-8da2-8e6faec3eef7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:16.864870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e82c-e20f-429f-8da2-8e6faec3eef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:16.884830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-4003e82c-e20f-429f-8da2-8e6faec3eef7 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:16.886051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4003e82c-e20f-429f-8da2-8e6faec3eef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:16.894269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:17.049507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46f1fe-fa93-4338-bed8-6e7f80e9ba39 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:17.064377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46f1fe-fa93-4338-bed8-6e7f80e9ba39 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:17.070109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67fc91-eac8-4cca-9004-dee57fd3e1c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:17.071205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67fc91-eac8-4cca-9004-dee57fd3e1c0 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:17.071663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67fc91-eac8-4cca-9004-dee57fd3e1c0 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:17.071932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67fc91-eac8-4cca-9004-dee57fd3e1c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:17.080864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67fc91-eac8-4cca-9004-dee57fd3e1c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:17.082477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67fc91-eac8-4cca-9004-dee57fd3e1c0 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:17.093290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 763/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:17.918388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa0acb-6f9a-4a39-b81f-c3092eec42fc None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:17.918906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa0acb-6f9a-4a39-b81f-c3092eec42fc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:17.918906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa0acb-6f9a-4a39-b81f-c3092eec42fc None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:17.920216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 16:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:58:17.925289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691608ac-b665-4e3c-8630-5633a41e2f6d None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:17.926038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691608ac-b665-4e3c-8630-5633a41e2f6d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:17.926140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691608ac-b665-4e3c-8630-5633a41e2f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:17.945505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-691608ac-b665-4e3c-8630-5633a41e2f6d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:17.946749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-691608ac-b665-4e3c-8630-5633a41e2f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:17.955571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 16:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:18.108455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa0acb-6f9a-4a39-b81f-c3092eec42fc None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:18.123386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa0acb-6f9a-4a39-b81f-c3092eec42fc None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:18.128978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591568c-5b3d-48fa-a05e-5c039c7524ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:18.129572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591568c-5b3d-48fa-a05e-5c039c7524ff service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:18.129880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591568c-5b3d-48fa-a05e-5c039c7524ff service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:18.130151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591568c-5b3d-48fa-a05e-5c039c7524ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:18.138847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591568c-5b3d-48fa-a05e-5c039c7524ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:18.140431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591568c-5b3d-48fa-a05e-5c039c7524ff service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:18.150530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 769/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:22.121438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30ce53-5c82-40a4-b6ea-b4cb59e8870a None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:22.122061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30ce53-5c82-40a4-b6ea-b4cb59e8870a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:22.122061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30ce53-5c82-40a4-b6ea-b4cb59e8870a None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:22.123401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:58:22.128677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a329e6-e9a3-4fe2-86bd-8e1eaa69c6b5 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:22.129127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a329e6-e9a3-4fe2-86bd-8e1eaa69c6b5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:22.129257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a329e6-e9a3-4fe2-86bd-8e1eaa69c6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:22.148282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f7a329e6-e9a3-4fe2-86bd-8e1eaa69c6b5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:22.149472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f7a329e6-e9a3-4fe2-86bd-8e1eaa69c6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:22.157187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:22.231953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30ce53-5c82-40a4-b6ea-b4cb59e8870a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:22.233689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-bd30ce53-5c82-40a4-b6ea-b4cb59e8870a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:22.317893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30ce53-5c82-40a4-b6ea-b4cb59e8870a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:22.320113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d077b-26a4-4bad-925a-f0d065498141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:22.320727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d077b-26a4-4bad-925a-f0d065498141 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:22.321003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d077b-26a4-4bad-925a-f0d065498141 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:22.321270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d077b-26a4-4bad-925a-f0d065498141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:22.330676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d077b-26a4-4bad-925a-f0d065498141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:22.332139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d077b-26a4-4bad-925a-f0d065498141 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:22.342314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 766/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:24.418086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46ca3b-63c7-4f9d-84db-881ecd27968d None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:24.418635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46ca3b-63c7-4f9d-84db-881ecd27968d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:24.418635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46ca3b-63c7-4f9d-84db-881ecd27968d None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:24.419872 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:58:24.425646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1d7d5-c0b6-4b5a-928b-0822757242b0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:24.426125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1d7d5-c0b6-4b5a-928b-0822757242b0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:24.426239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1d7d5-c0b6-4b5a-928b-0822757242b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:24.446518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b3a1d7d5-c0b6-4b5a-928b-0822757242b0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:24.447755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b3a1d7d5-c0b6-4b5a-928b-0822757242b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:24.455576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:24.491953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46ca3b-63c7-4f9d-84db-881ecd27968d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:24.503431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46ca3b-63c7-4f9d-84db-881ecd27968d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:24.507386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbf616-b7bd-4228-8685-02298e7b57af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:24.508543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbf616-b7bd-4228-8685-02298e7b57af service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:24.508827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbf616-b7bd-4228-8685-02298e7b57af service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:24.509100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbf616-b7bd-4228-8685-02298e7b57af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:24.518661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbf616-b7bd-4228-8685-02298e7b57af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:24.520114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbf616-b7bd-4228-8685-02298e7b57af service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:24.530300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 772/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:28.156271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d01f27-f11e-416a-9728-1f6144da632c None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:28.156851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d01f27-f11e-416a-9728-1f6144da632c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:28.156851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d01f27-f11e-416a-9728-1f6144da632c None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:28.158215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 768/1540] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:58:28.164151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b545bb2-7103-4cdd-8ec3-59de4bbc80bb None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:28.164674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b545bb2-7103-4cdd-8ec3-59de4bbc80bb None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:28.164674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b545bb2-7103-4cdd-8ec3-59de4bbc80bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:28.184643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-4b545bb2-7103-4cdd-8ec3-59de4bbc80bb None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:28.185876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4b545bb2-7103-4cdd-8ec3-59de4bbc80bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:28.195873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 773/1541] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:28.507687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d01f27-f11e-416a-9728-1f6144da632c None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:28.510014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-09d01f27-f11e-416a-9728-1f6144da632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:28.514156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=83 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:28.592097 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d01f27-f11e-416a-9728-1f6144da632c None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:28.596884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2154fc6-963d-4837-83e5-5bff280c0906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:28.597493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2154fc6-963d-4837-83e5-5bff280c0906 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:28.597766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2154fc6-963d-4837-83e5-5bff280c0906 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:28.598062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2154fc6-963d-4837-83e5-5bff280c0906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:28.607514 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2154fc6-963d-4837-83e5-5bff280c0906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:28.608954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2154fc6-963d-4837-83e5-5bff280c0906 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:28.619080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 769/1542] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:30.687569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63acc24-a1e2-4602-b02e-60aa8646db02 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:30.688146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63acc24-a1e2-4602-b02e-60aa8646db02 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:30.688146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63acc24-a1e2-4602-b02e-60aa8646db02 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:30.689433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 774/1543] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 16:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 16:58:30.694810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d8546-6327-45d0-9413-a54a186ae396 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:30.695301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d8546-6327-45d0-9413-a54a186ae396 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:30.695928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d8546-6327-45d0-9413-a54a186ae396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:30.716956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-243d8546-6327-45d0-9413-a54a186ae396 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 16:58:30.718227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-243d8546-6327-45d0-9413-a54a186ae396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:30.726287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 770/1544] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 16:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 16:58:31.032170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63acc24-a1e2-4602-b02e-60aa8646db02 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 16:58:31.033889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b63acc24-a1e2-4602-b02e-60aa8646db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 16:58:31.048469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63acc24-a1e2-4602-b02e-60aa8646db02 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 16:58:31.053057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb4c42-82bf-40b9-92b4-f621c4fbe8b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 16:58:31.053647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb4c42-82bf-40b9-92b4-f621c4fbe8b2 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 16:58:31.053976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb4c42-82bf-40b9-92b4-f621c4fbe8b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 16:58:31.054244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb4c42-82bf-40b9-92b4-f621c4fbe8b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 16:58:31.064164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb4c42-82bf-40b9-92b4-f621c4fbe8b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 16:58:31.065584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb4c42-82bf-40b9-92b4-f621c4fbe8b2 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 16:58:31.075195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 775/1545] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 16:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 16:58:40.710696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=37 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 16:58:41.071462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=42 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:01.051391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc02c0-9441-4153-8b33-de0316633009 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.052188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc02c0-9441-4153-8b33-de0316633009 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.052188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc02c0-9441-4153-8b33-de0316633009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.080141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b0fc02c0-9441-4153-8b33-de0316633009 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:01.081563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b0fc02c0-9441-4153-8b33-de0316633009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.094378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 771/1546] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:01.104633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34cc6e-de29-4e74-803a-6872b7536378 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.105209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34cc6e-de29-4e74-803a-6872b7536378 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.105209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34cc6e-de29-4e74-803a-6872b7536378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.130084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-7f34cc6e-de29-4e74-803a-6872b7536378 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:01.131315 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7f34cc6e-de29-4e74-803a-6872b7536378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.145073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 776/1547] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:01.150136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc02c0-9441-4153-8b33-de0316633009 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:01.161125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc02c0-9441-4153-8b33-de0316633009 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:01.174083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f75d2-976b-40f5-a4a8-0b1acbfb2f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:01.174728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f75d2-976b-40f5-a4a8-0b1acbfb2f43 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.175011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f75d2-976b-40f5-a4a8-0b1acbfb2f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.175282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f75d2-976b-40f5-a4a8-0b1acbfb2f43 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.175944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f75d2-976b-40f5-a4a8-0b1acbfb2f43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:01.176253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f75d2-976b-40f5-a4a8-0b1acbfb2f43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.177911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f75d2-976b-40f5-a4a8-0b1acbfb2f43 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:01.192350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 772/1548] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:01.201221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34cc6e-de29-4e74-803a-6872b7536378 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:01.211884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34cc6e-de29-4e74-803a-6872b7536378 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:01.218413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b18200-1967-49e6-a28b-ffac2db51427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:01.219022 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b18200-1967-49e6-a28b-ffac2db51427 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.219323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b18200-1967-49e6-a28b-ffac2db51427 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.219559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b18200-1967-49e6-a28b-ffac2db51427 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.220191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b18200-1967-49e6-a28b-ffac2db51427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.222136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b18200-1967-49e6-a28b-ffac2db51427 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:01.381288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 777/1549] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:01.386059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f75d2-976b-40f5-a4a8-0b1acbfb2f43 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:01.397523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f75d2-976b-40f5-a4a8-0b1acbfb2f43 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:01.467127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28a79c-c9bc-4801-b7d3-bf8d5c1403f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:01.467848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28a79c-c9bc-4801-b7d3-bf8d5c1403f6 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.468132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28a79c-c9bc-4801-b7d3-bf8d5c1403f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.468403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28a79c-c9bc-4801-b7d3-bf8d5c1403f6 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.469037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28a79c-c9bc-4801-b7d3-bf8d5c1403f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.469345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7658f58-3cf9-443f-8979-a547ca144627 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.469633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7658f58-3cf9-443f-8979-a547ca144627 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.469984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7658f58-3cf9-443f-8979-a547ca144627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.471003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28a79c-c9bc-4801-b7d3-bf8d5c1403f6 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:01.522942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c7658f58-3cf9-443f-8979-a547ca144627 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:01.524711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c7658f58-3cf9-443f-8979-a547ca144627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.544048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 778/1550] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:01.548887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70f2bc-83c1-424a-965a-d68c65adae28 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.551931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70f2bc-83c1-424a-965a-d68c65adae28 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.551931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70f2bc-83c1-424a-965a-d68c65adae28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.579049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9a70f2bc-83c1-424a-965a-d68c65adae28 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:01.580296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9a70f2bc-83c1-424a-965a-d68c65adae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.593325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 773/1551] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:01.597736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 779/1552] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:00:01.597736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b892c85-6d3b-4ddc-9035-2d55a825b8b8 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.597736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b892c85-6d3b-4ddc-9035-2d55a825b8b8 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.597736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b892c85-6d3b-4ddc-9035-2d55a825b8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.601020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28a79c-c9bc-4801-b7d3-bf8d5c1403f6 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:01.616841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28a79c-c9bc-4801-b7d3-bf8d5c1403f6 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:01.639347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb511a-fcf6-41e2-8ac3-435536174ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:01.639566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0b892c85-6d3b-4ddc-9035-2d55a825b8b8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:01.639891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb511a-fcf6-41e2-8ac3-435536174ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.640172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb511a-fcf6-41e2-8ac3-435536174ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.640442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb511a-fcf6-41e2-8ac3-435536174ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.640748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0b892c85-6d3b-4ddc-9035-2d55a825b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.641235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb511a-fcf6-41e2-8ac3-435536174ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.642892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb511a-fcf6-41e2-8ac3-435536174ef5 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:01.654668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 780/1553] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:01.656170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59809d3e-dc32-4e5b-bea3-54e9d2bc8ed2 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.656467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59809d3e-dc32-4e5b-bea3-54e9d2bc8ed2 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.656717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59809d3e-dc32-4e5b-bea3-54e9d2bc8ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.665881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 774/1554] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:01.667998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce846fdb-861e-4d35-80a3-24401e502548 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.668502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce846fdb-861e-4d35-80a3-24401e502548 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.668991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce846fdb-861e-4d35-80a3-24401e502548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.702928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-59809d3e-dc32-4e5b-bea3-54e9d2bc8ed2 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:01.704145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-59809d3e-dc32-4e5b-bea3-54e9d2bc8ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.711465 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ce846fdb-861e-4d35-80a3-24401e502548 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:01.713265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ce846fdb-861e-4d35-80a3-24401e502548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.723965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 781/1555] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:01.724879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59809d3e-dc32-4e5b-bea3-54e9d2bc8ed2 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:01.734119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 775/1556] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:01.735486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce846fdb-861e-4d35-80a3-24401e502548 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:01.742373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59809d3e-dc32-4e5b-bea3-54e9d2bc8ed2 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:01.751918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:01.752541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.752816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.753083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.753687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce846fdb-861e-4d35-80a3-24401e502548 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:01.753845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:01.754050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.757272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: WARNING py.warnings [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 17:00:01.757272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: warnings.warn(msg) Aug 08 17:00:01.757272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: Aug 08 17:00:01.757272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:01.771373 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f551c-8ead-4d49-a1bb-8f8365f2e61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:01.772371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f551c-8ead-4d49-a1bb-8f8365f2e61b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.772851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f551c-8ead-4d49-a1bb-8f8365f2e61b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.773401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f551c-8ead-4d49-a1bb-8f8365f2e61b admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.774975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f551c-8ead-4d49-a1bb-8f8365f2e61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.776810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f551c-8ead-4d49-a1bb-8f8365f2e61b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:01.782336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 782/1557] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:01.783211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:01.794535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998514a-5426-4900-8649-4ed64a84e11f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:01.802997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 776/1558] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:01.805237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731485f-1f25-43ff-a384-b4a8bb830a1a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.805708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1bbf2-5810-483d-b6f4-335d4d4b31fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:01.806047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731485f-1f25-43ff-a384-b4a8bb830a1a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.806516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731485f-1f25-43ff-a384-b4a8bb830a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.806634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1bbf2-5810-483d-b6f4-335d4d4b31fc admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.806897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1bbf2-5810-483d-b6f4-335d4d4b31fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.807194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1bbf2-5810-483d-b6f4-335d4d4b31fc admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.807849 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1bbf2-5810-483d-b6f4-335d4d4b31fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:01.808162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1bbf2-5810-483d-b6f4-335d4d4b31fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.809803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1bbf2-5810-483d-b6f4-335d4d4b31fc admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:01.830551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 783/1559] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:01.831915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bfe28-f61c-471a-8488-790575b518f0 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.832204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bfe28-f61c-471a-8488-790575b518f0 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.832475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bfe28-f61c-471a-8488-790575b518f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.839950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5731485f-1f25-43ff-a384-b4a8bb830a1a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:01.847822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5731485f-1f25-43ff-a384-b4a8bb830a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.851783 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 777/1560] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:01.852950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731485f-1f25-43ff-a384-b4a8bb830a1a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:01.855466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-143bfe28-f61c-471a-8488-790575b518f0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:01.856675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-143bfe28-f61c-471a-8488-790575b518f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:01.865142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731485f-1f25-43ff-a384-b4a8bb830a1a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:01.866340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 784/1561] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:01.867177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bfe28-f61c-471a-8488-790575b518f0 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:01.874438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a3e25-46d2-48e9-8332-ab9a9d7421b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:01.875368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a3e25-46d2-48e9-8332-ab9a9d7421b3 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.875715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a3e25-46d2-48e9-8332-ab9a9d7421b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.876049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a3e25-46d2-48e9-8332-ab9a9d7421b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.876737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a3e25-46d2-48e9-8332-ab9a9d7421b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.877814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bfe28-f61c-471a-8488-790575b518f0 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:01.878116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING py.warnings [None req-b00a3e25-46d2-48e9-8332-ab9a9d7421b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 17:00:01.878116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: warnings.warn(msg) Aug 08 17:00:01.878116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: Aug 08 17:00:01.879589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a3e25-46d2-48e9-8332-ab9a9d7421b3 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:01.882436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c5a20-8d96-42e9-a338-f7cc77cdc882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:01.883613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c5a20-8d96-42e9-a338-f7cc77cdc882 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:01.883888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c5a20-8d96-42e9-a338-f7cc77cdc882 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:01.884153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c5a20-8d96-42e9-a338-f7cc77cdc882 admin admin] PATH_INFO: `/v3/projects/8bc70ef2b50441e1baf52e05d005fe9a/users/908b1d1f3c9f42e699680b98e3e4b360/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:01.900682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c5a20-8d96-42e9-a338-f7cc77cdc882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc70ef2b50441e1baf52e05d005fe9a, user_id=908b1d1f3c9f42e699680b98e3e4b360, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:01.902235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c5a20-8d96-42e9-a338-f7cc77cdc882 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.030681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 778/1562] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:01 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:02.032180 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030c27-0b29-4ab6-9dfe-58bebe32094a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.032534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030c27-0b29-4ab6-9dfe-58bebe32094a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.032868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030c27-0b29-4ab6-9dfe-58bebe32094a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.052872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b8030c27-0b29-4ab6-9dfe-58bebe32094a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:02.054123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b8030c27-0b29-4ab6-9dfe-58bebe32094a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:02.085657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 785/1563] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:01 2022] PUT /identity/v3/projects/8bc70ef2b50441e1baf52e05d005fe9a/users/908b1d1f3c9f42e699680b98e3e4b360/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:02.089839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c5a20-8d96-42e9-a338-f7cc77cdc882 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.124771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 779/1564] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:02.126132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f946632-4cb3-4855-bd25-b15b4fba5a5f None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.137112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f946632-4cb3-4855-bd25-b15b4fba5a5f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.137112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f946632-4cb3-4855-bd25-b15b4fba5a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.146489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8f946632-4cb3-4855-bd25-b15b4fba5a5f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:02.147706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8f946632-4cb3-4855-bd25-b15b4fba5a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:02.153010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c5a20-8d96-42e9-a338-f7cc77cdc882 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.155776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 780/1565] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:02.156616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f946632-4cb3-4855-bd25-b15b4fba5a5f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.157707 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c3015-021b-43c2-b70f-8a50f5527afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.158403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3015-021b-43c2-b70f-8a50f5527afc admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.158676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3015-021b-43c2-b70f-8a50f5527afc admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.158941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c3015-021b-43c2-b70f-8a50f5527afc admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.159557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c3015-021b-43c2-b70f-8a50f5527afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.161556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c3015-021b-43c2-b70f-8a50f5527afc admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.168855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f946632-4cb3-4855-bd25-b15b4fba5a5f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.174576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eff96-5c72-45b3-830c-2d3fb2f6dfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.181119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eff96-5c72-45b3-830c-2d3fb2f6dfcb admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.181119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eff96-5c72-45b3-830c-2d3fb2f6dfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.181119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eff96-5c72-45b3-830c-2d3fb2f6dfcb admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.181119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eff96-5c72-45b3-830c-2d3fb2f6dfcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:02.181119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eff96-5c72-45b3-830c-2d3fb2f6dfcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.181119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eff96-5c72-45b3-830c-2d3fb2f6dfcb admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.181685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:02.191434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 781/1566] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:02.192892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a68b8b-9833-4218-8b0d-381d0a64fad8 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.193186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a68b8b-9833-4218-8b0d-381d0a64fad8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.193521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a68b8b-9833-4218-8b0d-381d0a64fad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.213313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-80a68b8b-9833-4218-8b0d-381d0a64fad8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:02.214572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-80a68b8b-9833-4218-8b0d-381d0a64fad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:02.222754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 782/1567] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:02.223942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830090d-0ae1-498b-87fb-d592f8b22ff4 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.224227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830090d-0ae1-498b-87fb-d592f8b22ff4 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.224503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830090d-0ae1-498b-87fb-d592f8b22ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.248260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-0830090d-0ae1-498b-87fb-d592f8b22ff4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:02.249504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0830090d-0ae1-498b-87fb-d592f8b22ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:02.257520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 783/1568] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:02.258861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830090d-0ae1-498b-87fb-d592f8b22ff4 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.268531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830090d-0ae1-498b-87fb-d592f8b22ff4 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.273262 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c2eb1-cb25-4b3c-902b-187939686630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.273906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c2eb1-cb25-4b3c-902b-187939686630 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.274189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c2eb1-cb25-4b3c-902b-187939686630 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.274460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c2eb1-cb25-4b3c-902b-187939686630 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.275081 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c2eb1-cb25-4b3c-902b-187939686630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.277062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c2eb1-cb25-4b3c-902b-187939686630 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.323609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 786/1569] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/projects => generated 373 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:02.325035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd553bb9-e202-4283-9b8a-0d8a89c16777 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.325397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd553bb9-e202-4283-9b8a-0d8a89c16777 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.325749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd553bb9-e202-4283-9b8a-0d8a89c16777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.353219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-cd553bb9-e202-4283-9b8a-0d8a89c16777 None None] MFA Rules not processed for user `908b1d1f3c9f42e699680b98e3e4b360`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:02.354640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-cd553bb9-e202-4283-9b8a-0d8a89c16777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:02.368080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:02.421820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 787/1570] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:02.423278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed83d3-2f6b-4962-abe5-2e223d1ee4ad None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.423556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed83d3-2f6b-4962-abe5-2e223d1ee4ad None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.423830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed83d3-2f6b-4962-abe5-2e223d1ee4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.432530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 784/1571] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:02.433871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164eff7-7e95-4e18-981d-5431b7acfae6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.434186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164eff7-7e95-4e18-981d-5431b7acfae6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.434453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164eff7-7e95-4e18-981d-5431b7acfae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.447230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-10ed83d3-2f6b-4962-abe5-2e223d1ee4ad None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:02.448442 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-10ed83d3-2f6b-4962-abe5-2e223d1ee4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:02.453323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-0164eff7-7e95-4e18-981d-5431b7acfae6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:02.454613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0164eff7-7e95-4e18-981d-5431b7acfae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:02.516324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 788/1572] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:02.517701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed83d3-2f6b-4962-abe5-2e223d1ee4ad None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.526011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 785/1573] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:02.527114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164eff7-7e95-4e18-981d-5431b7acfae6 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.527258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed83d3-2f6b-4962-abe5-2e223d1ee4ad None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.535129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfc2a6-2f73-41c9-a376-3dc680f22e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.535129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfc2a6-2f73-41c9-a376-3dc680f22e3e admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.535129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfc2a6-2f73-41c9-a376-3dc680f22e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.535129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfc2a6-2f73-41c9-a376-3dc680f22e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.535554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfc2a6-2f73-41c9-a376-3dc680f22e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.536767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfc2a6-2f73-41c9-a376-3dc680f22e3e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.539625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164eff7-7e95-4e18-981d-5431b7acfae6 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.544790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb9d52-55bb-4741-a1e4-a67b4305f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.545383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb9d52-55bb-4741-a1e4-a67b4305f2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.545666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb9d52-55bb-4741-a1e4-a67b4305f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.546198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb9d52-55bb-4741-a1e4-a67b4305f2d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.546935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb9d52-55bb-4741-a1e4-a67b4305f2d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:02.547236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb9d52-55bb-4741-a1e4-a67b4305f2d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.548818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb9d52-55bb-4741-a1e4-a67b4305f2d2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.561334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 786/1574] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:02.562720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d589ab6-89e2-4f9c-b888-0db3ea3e2de8 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.563003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d589ab6-89e2-4f9c-b888-0db3ea3e2de8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.563269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d589ab6-89e2-4f9c-b888-0db3ea3e2de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.582552 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-2d589ab6-89e2-4f9c-b888-0db3ea3e2de8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:02.591168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2d589ab6-89e2-4f9c-b888-0db3ea3e2de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:02.692282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 787/1575] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:02.692282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d589ab6-89e2-4f9c-b888-0db3ea3e2de8 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.692282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d589ab6-89e2-4f9c-b888-0db3ea3e2de8 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.692282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b893a3-7a91-4428-a822-b9acc0075bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.692681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b893a3-7a91-4428-a822-b9acc0075bad admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.692681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b893a3-7a91-4428-a822-b9acc0075bad admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.692681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b893a3-7a91-4428-a822-b9acc0075bad admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.692681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b893a3-7a91-4428-a822-b9acc0075bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.692681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b893a3-7a91-4428-a822-b9acc0075bad admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.740665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 789/1576] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:02.747493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfc2a6-2f73-41c9-a376-3dc680f22e3e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.801916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfc2a6-2f73-41c9-a376-3dc680f22e3e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.807038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf964e1f-2666-484e-9fa7-32d6a6399055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.807691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf964e1f-2666-484e-9fa7-32d6a6399055 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.808193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf964e1f-2666-484e-9fa7-32d6a6399055 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.809396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf964e1f-2666-484e-9fa7-32d6a6399055 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.810145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf964e1f-2666-484e-9fa7-32d6a6399055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.812307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf964e1f-2666-484e-9fa7-32d6a6399055 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.816795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 788/1577] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:02.816795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b893a3-7a91-4428-a822-b9acc0075bad admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.825014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b893a3-7a91-4428-a822-b9acc0075bad admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.830460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c0831-98ac-4dfa-b398-00a8ae789bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.831077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c0831-98ac-4dfa-b398-00a8ae789bad admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.831382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c0831-98ac-4dfa-b398-00a8ae789bad admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.831654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c0831-98ac-4dfa-b398-00a8ae789bad admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.832297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c0831-98ac-4dfa-b398-00a8ae789bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.833940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c0831-98ac-4dfa-b398-00a8ae789bad admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.845872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 789/1578] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:02.846552 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c0831-98ac-4dfa-b398-00a8ae789bad admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.856852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c0831-98ac-4dfa-b398-00a8ae789bad admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.861458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353369-e96a-497b-8d00-ee4946b249a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.862136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353369-e96a-497b-8d00-ee4946b249a9 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.862417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353369-e96a-497b-8d00-ee4946b249a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.862685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353369-e96a-497b-8d00-ee4946b249a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.863312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353369-e96a-497b-8d00-ee4946b249a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:02.863615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353369-e96a-497b-8d00-ee4946b249a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.865280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353369-e96a-497b-8d00-ee4946b249a9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.877087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 790/1579] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:02.878232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353369-e96a-497b-8d00-ee4946b249a9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.887598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353369-e96a-497b-8d00-ee4946b249a9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.892322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa36890-b8af-4bdb-9367-405abd3b38e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.892892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa36890-b8af-4bdb-9367-405abd3b38e4 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.893169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa36890-b8af-4bdb-9367-405abd3b38e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.893502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa36890-b8af-4bdb-9367-405abd3b38e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.894167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa36890-b8af-4bdb-9367-405abd3b38e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:02.894467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa36890-b8af-4bdb-9367-405abd3b38e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.896179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa36890-b8af-4bdb-9367-405abd3b38e4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:02.908582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 791/1580] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:02.909473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa36890-b8af-4bdb-9367-405abd3b38e4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:02.920012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa36890-b8af-4bdb-9367-405abd3b38e4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:02.931335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165298c-69ff-49f0-8c46-4debaec0de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:02.931335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165298c-69ff-49f0-8c46-4debaec0de56 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:02.931335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165298c-69ff-49f0-8c46-4debaec0de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:02.931335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165298c-69ff-49f0-8c46-4debaec0de56 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:02.931786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165298c-69ff-49f0-8c46-4debaec0de56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:02.931786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165298c-69ff-49f0-8c46-4debaec0de56 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.003182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 790/1581] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/projects => generated 369 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.006849 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf964e1f-2666-484e-9fa7-32d6a6399055 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.027056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:03.035574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 792/1582] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:02 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.036613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165298c-69ff-49f0-8c46-4debaec0de56 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.067257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:03.070740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf964e1f-2666-484e-9fa7-32d6a6399055 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.099260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165298c-69ff-49f0-8c46-4debaec0de56 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.104826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45137e82-332f-4e5e-83dc-8ae84f158f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.105433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45137e82-332f-4e5e-83dc-8ae84f158f4b admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.105709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45137e82-332f-4e5e-83dc-8ae84f158f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.106008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45137e82-332f-4e5e-83dc-8ae84f158f4b admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.106628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45137e82-332f-4e5e-83dc-8ae84f158f4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.108550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45137e82-332f-4e5e-83dc-8ae84f158f4b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.128162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fc7f7-2781-4a40-b7ac-522096511a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.128759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fc7f7-2781-4a40-b7ac-522096511a4a admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.129039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fc7f7-2781-4a40-b7ac-522096511a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.129306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fc7f7-2781-4a40-b7ac-522096511a4a admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.129989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fc7f7-2781-4a40-b7ac-522096511a4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:03.130308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fc7f7-2781-4a40-b7ac-522096511a4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.132028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fc7f7-2781-4a40-b7ac-522096511a4a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.143500 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 791/1583] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:03.144275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fc7f7-2781-4a40-b7ac-522096511a4a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.154017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fc7f7-2781-4a40-b7ac-522096511a4a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.158619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a011b-27ca-4d2d-91d2-4be4228830f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.159201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a011b-27ca-4d2d-91d2-4be4228830f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.159477 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a011b-27ca-4d2d-91d2-4be4228830f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.159746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a011b-27ca-4d2d-91d2-4be4228830f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.160375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a011b-27ca-4d2d-91d2-4be4228830f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.161985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a011b-27ca-4d2d-91d2-4be4228830f5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.173057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 792/1584] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:03.173864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a011b-27ca-4d2d-91d2-4be4228830f5 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.184769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a011b-27ca-4d2d-91d2-4be4228830f5 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.189334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93c340-fe4b-49f3-8047-42f147e1003b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.189955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c340-fe4b-49f3-8047-42f147e1003b admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.190235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c340-fe4b-49f3-8047-42f147e1003b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.190503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c340-fe4b-49f3-8047-42f147e1003b admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.191217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93c340-fe4b-49f3-8047-42f147e1003b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.192936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93c340-fe4b-49f3-8047-42f147e1003b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.202961 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 793/1585] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:03.203791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93c340-fe4b-49f3-8047-42f147e1003b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.213518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 793/1586] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:03 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.217933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93c340-fe4b-49f3-8047-42f147e1003b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.218068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45137e82-332f-4e5e-83dc-8ae84f158f4b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.218917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6e243-154f-498a-9606-3178f42a7541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.219513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6e243-154f-498a-9606-3178f42a7541 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.220029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6e243-154f-498a-9606-3178f42a7541 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.220303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6e243-154f-498a-9606-3178f42a7541 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.220973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6e243-154f-498a-9606-3178f42a7541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.222914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6e243-154f-498a-9606-3178f42a7541 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.224369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45137e82-332f-4e5e-83dc-8ae84f158f4b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.229538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df562e-d1a6-4a6f-bb2c-efdb2200bb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.230192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df562e-d1a6-4a6f-bb2c-efdb2200bb21 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.230467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df562e-d1a6-4a6f-bb2c-efdb2200bb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.230739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df562e-d1a6-4a6f-bb2c-efdb2200bb21 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.231351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df562e-d1a6-4a6f-bb2c-efdb2200bb21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.233572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df562e-d1a6-4a6f-bb2c-efdb2200bb21 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.320946 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 794/1587] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:03 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.320946 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6e243-154f-498a-9606-3178f42a7541 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.346683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 794/1588] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:03 2022] POST /identity/v3/projects => generated 405 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.347502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df562e-d1a6-4a6f-bb2c-efdb2200bb21 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.385637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6e243-154f-498a-9606-3178f42a7541 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.405148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:03.410799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df562e-d1a6-4a6f-bb2c-efdb2200bb21 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.416199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc7068-a179-4e02-89a4-0b2ca09fbb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.420198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc7068-a179-4e02-89a4-0b2ca09fbb79 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.420198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc7068-a179-4e02-89a4-0b2ca09fbb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.420198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc7068-a179-4e02-89a4-0b2ca09fbb79 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.420198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc7068-a179-4e02-89a4-0b2ca09fbb79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.420198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc7068-a179-4e02-89a4-0b2ca09fbb79 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.444442 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba5e7e-a627-4478-b611-ec5c22f2dbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.445096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba5e7e-a627-4478-b611-ec5c22f2dbd4 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.445377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba5e7e-a627-4478-b611-ec5c22f2dbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.445647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba5e7e-a627-4478-b611-ec5c22f2dbd4 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.446358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba5e7e-a627-4478-b611-ec5c22f2dbd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.447904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: WARNING py.warnings [None req-73ba5e7e-a627-4478-b611-ec5c22f2dbd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 17:00:03.447904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: warnings.warn(msg) Aug 08 17:00:03.447904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: Aug 08 17:00:03.451533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba5e7e-a627-4478-b611-ec5c22f2dbd4 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.565595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 795/1589] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:03 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.566513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc7068-a179-4e02-89a4-0b2ca09fbb79 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.576430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc7068-a179-4e02-89a4-0b2ca09fbb79 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.581034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb85db9-275b-40c6-8b63-68e90e6186d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.581632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb85db9-275b-40c6-8b63-68e90e6186d8 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.581942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb85db9-275b-40c6-8b63-68e90e6186d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.582214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb85db9-275b-40c6-8b63-68e90e6186d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.582839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb85db9-275b-40c6-8b63-68e90e6186d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.587123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 795/1590] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:03 2022] POST /identity/v3/users => generated 291 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.587312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb85db9-275b-40c6-8b63-68e90e6186d8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.587398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba5e7e-a627-4478-b611-ec5c22f2dbd4 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.598839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba5e7e-a627-4478-b611-ec5c22f2dbd4 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.604426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09ada1-74e9-4b13-ba67-7b5b2ea8a006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.605035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09ada1-74e9-4b13-ba67-7b5b2ea8a006 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.605312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09ada1-74e9-4b13-ba67-7b5b2ea8a006 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.605581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09ada1-74e9-4b13-ba67-7b5b2ea8a006 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.606263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09ada1-74e9-4b13-ba67-7b5b2ea8a006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.607950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09ada1-74e9-4b13-ba67-7b5b2ea8a006 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.618716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 796/1591] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:03.619659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09ada1-74e9-4b13-ba67-7b5b2ea8a006 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.631588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09ada1-74e9-4b13-ba67-7b5b2ea8a006 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.636448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947a467-abc5-4638-87f8-e98797c0021b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.637101 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947a467-abc5-4638-87f8-e98797c0021b admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.637378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947a467-abc5-4638-87f8-e98797c0021b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.637652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947a467-abc5-4638-87f8-e98797c0021b admin admin] PATH_INFO: `/v3/projects/db9c728a8d56493db7bde4b0bc36f550/users/ae31b2903c8e458ca7c68082572db8f4/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.653353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947a467-abc5-4638-87f8-e98797c0021b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9c728a8d56493db7bde4b0bc36f550, user_id=ae31b2903c8e458ca7c68082572db8f4, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.656074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947a467-abc5-4638-87f8-e98797c0021b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.696068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 796/1592] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:03 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.696068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb85db9-275b-40c6-8b63-68e90e6186d8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.760457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb85db9-275b-40c6-8b63-68e90e6186d8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.765187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29882d-e3cc-410d-8b2c-47d8cfc68401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.765912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29882d-e3cc-410d-8b2c-47d8cfc68401 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.766190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29882d-e3cc-410d-8b2c-47d8cfc68401 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.766454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29882d-e3cc-410d-8b2c-47d8cfc68401 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.767101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29882d-e3cc-410d-8b2c-47d8cfc68401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.768685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29882d-e3cc-410d-8b2c-47d8cfc68401 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.771094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:03.780579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 797/1593] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:03.781447 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29882d-e3cc-410d-8b2c-47d8cfc68401 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.792218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 797/1594] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:03 2022] PUT /identity/v3/projects/db9c728a8d56493db7bde4b0bc36f550/users/ae31b2903c8e458ca7c68082572db8f4/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:03.792456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29882d-e3cc-410d-8b2c-47d8cfc68401 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.793136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947a467-abc5-4638-87f8-e98797c0021b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.860736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eab20-76a7-4458-b1af-6f9b9b840fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.861258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eab20-76a7-4458-b1af-6f9b9b840fdb admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.861539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eab20-76a7-4458-b1af-6f9b9b840fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.861821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eab20-76a7-4458-b1af-6f9b9b840fdb admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.862449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eab20-76a7-4458-b1af-6f9b9b840fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.862584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947a467-abc5-4638-87f8-e98797c0021b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:03.864506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eab20-76a7-4458-b1af-6f9b9b840fdb admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.868257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b0caa-f7c4-4a9c-8795-1cea0c333a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:03.869203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b0caa-f7c4-4a9c-8795-1cea0c333a0c admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:03.869955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b0caa-f7c4-4a9c-8795-1cea0c333a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:03.870404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b0caa-f7c4-4a9c-8795-1cea0c333a0c admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:03.871180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b0caa-f7c4-4a9c-8795-1cea0c333a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:03.873325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b0caa-f7c4-4a9c-8795-1cea0c333a0c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:03.990698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 798/1595] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:03 2022] POST /identity/v3/users => generated 303 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.991247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 798/1596] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:03 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:03.991656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b0caa-f7c4-4a9c-8795-1cea0c333a0c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:03.992245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eab20-76a7-4458-b1af-6f9b9b840fdb admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.002888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b0caa-f7c4-4a9c-8795-1cea0c333a0c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.003100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eab20-76a7-4458-b1af-6f9b9b840fdb admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.008400 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0d535-6d34-4705-a9d6-8d2c241edc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.008576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e679db8-54f4-4654-b566-72454df56f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.009008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0d535-6d34-4705-a9d6-8d2c241edc9e admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.009117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e679db8-54f4-4654-b566-72454df56f7f admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.009282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0d535-6d34-4705-a9d6-8d2c241edc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.009379 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e679db8-54f4-4654-b566-72454df56f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.009554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0d535-6d34-4705-a9d6-8d2c241edc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.009662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e679db8-54f4-4654-b566-72454df56f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.010229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e679db8-54f4-4654-b566-72454df56f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.010353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0d535-6d34-4705-a9d6-8d2c241edc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.011855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0d535-6d34-4705-a9d6-8d2c241edc9e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.011978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e679db8-54f4-4654-b566-72454df56f7f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.022595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 799/1597] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.022914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 799/1598] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.024168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e679db8-54f4-4654-b566-72454df56f7f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.024314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0d535-6d34-4705-a9d6-8d2c241edc9e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.034949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0d535-6d34-4705-a9d6-8d2c241edc9e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.035166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e679db8-54f4-4654-b566-72454df56f7f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.039583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80ca38-c40c-4f2e-88ac-6ed66a1eb4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.039755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d8b1a-58be-43f9-9ca3-6057bb8eae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.040202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80ca38-c40c-4f2e-88ac-6ed66a1eb4a6 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.040353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d8b1a-58be-43f9-9ca3-6057bb8eae24 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.040460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80ca38-c40c-4f2e-88ac-6ed66a1eb4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.040555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d8b1a-58be-43f9-9ca3-6057bb8eae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.040727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80ca38-c40c-4f2e-88ac-6ed66a1eb4a6 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.040834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d8b1a-58be-43f9-9ca3-6057bb8eae24 admin admin] PATH_INFO: `/v3/projects/a326b1ccd4d3402d9abdd4dcbf6e3f2c/users/7127a5d0909a49b9b8ee4c472ac28012/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.041445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80ca38-c40c-4f2e-88ac-6ed66a1eb4a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.043406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80ca38-c40c-4f2e-88ac-6ed66a1eb4a6 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.056900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d8b1a-58be-43f9-9ca3-6057bb8eae24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a326b1ccd4d3402d9abdd4dcbf6e3f2c, user_id=7127a5d0909a49b9b8ee4c472ac28012, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.059590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d8b1a-58be-43f9-9ca3-6057bb8eae24 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.165498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 800/1599] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:04 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:04.166293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80ca38-c40c-4f2e-88ac-6ed66a1eb4a6 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.176714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80ca38-c40c-4f2e-88ac-6ed66a1eb4a6 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.178052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 800/1600] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:04 2022] PUT /identity/v3/projects/a326b1ccd4d3402d9abdd4dcbf6e3f2c/users/7127a5d0909a49b9b8ee4c472ac28012/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:04.179869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170d619-c45b-4624-b0db-8057d758d4ad None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.186295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170d619-c45b-4624-b0db-8057d758d4ad None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.186295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170d619-c45b-4624-b0db-8057d758d4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.211573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0170d619-c45b-4624-b0db-8057d758d4ad None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:04.212873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0170d619-c45b-4624-b0db-8057d758d4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:04.238481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ca928-e53c-42c9-94e3-ef289470cfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.239171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ca928-e53c-42c9-94e3-ef289470cfdd admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.239586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ca928-e53c-42c9-94e3-ef289470cfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.239984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ca928-e53c-42c9-94e3-ef289470cfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.240969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ca928-e53c-42c9-94e3-ef289470cfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.243146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ca928-e53c-42c9-94e3-ef289470cfdd admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.244241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:04.255647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 801/1601] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.262700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ca928-e53c-42c9-94e3-ef289470cfdd admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.268705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ca928-e53c-42c9-94e3-ef289470cfdd admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.273987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d6262-2779-48bf-866c-6a6e5ba7ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.274264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d6262-2779-48bf-866c-6a6e5ba7ba4e admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.274541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d6262-2779-48bf-866c-6a6e5ba7ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.274808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d6262-2779-48bf-866c-6a6e5ba7ba4e admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.275440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d6262-2779-48bf-866c-6a6e5ba7ba4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.277036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d6262-2779-48bf-866c-6a6e5ba7ba4e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.280254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 801/1602] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:04.281172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170d619-c45b-4624-b0db-8057d758d4ad None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.291327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170d619-c45b-4624-b0db-8057d758d4ad None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.291503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 802/1603] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.292335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d6262-2779-48bf-866c-6a6e5ba7ba4e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.297230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fd38e-952e-4947-91d6-f55384aabb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.298141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fd38e-952e-4947-91d6-f55384aabb03 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.298416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fd38e-952e-4947-91d6-f55384aabb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.298691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fd38e-952e-4947-91d6-f55384aabb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.299344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fd38e-952e-4947-91d6-f55384aabb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.301095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fd38e-952e-4947-91d6-f55384aabb03 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.302531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d6262-2779-48bf-866c-6a6e5ba7ba4e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.308223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4418053-a0f6-478b-a2b9-c21904d56347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.308856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4418053-a0f6-478b-a2b9-c21904d56347 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.322268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4418053-a0f6-478b-a2b9-c21904d56347 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.322268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4418053-a0f6-478b-a2b9-c21904d56347 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.322268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4418053-a0f6-478b-a2b9-c21904d56347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.322268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4418053-a0f6-478b-a2b9-c21904d56347 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.322651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 802/1604] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.322651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fd38e-952e-4947-91d6-f55384aabb03 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.322817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 803/1605] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.323674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4418053-a0f6-478b-a2b9-c21904d56347 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.325807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fd38e-952e-4947-91d6-f55384aabb03 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.330658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1e181-dc7e-4f12-99c2-36973ed9d27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.331474 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1e181-dc7e-4f12-99c2-36973ed9d27f admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.331756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1e181-dc7e-4f12-99c2-36973ed9d27f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.332035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1e181-dc7e-4f12-99c2-36973ed9d27f admin admin] PATH_INFO: `/v3/projects/95d311c9e53e4b2f89a1b7a95dd44dce/users/793cffd0e0c04069ba94846af1139f2d/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.333318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4418053-a0f6-478b-a2b9-c21904d56347 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.338106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfbe54-19b6-446e-b54f-a39da8e09da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.338689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfbe54-19b6-446e-b54f-a39da8e09da7 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.338969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfbe54-19b6-446e-b54f-a39da8e09da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.339238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfbe54-19b6-446e-b54f-a39da8e09da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.339881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfbe54-19b6-446e-b54f-a39da8e09da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.341450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfbe54-19b6-446e-b54f-a39da8e09da7 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.347026 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1e181-dc7e-4f12-99c2-36973ed9d27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d311c9e53e4b2f89a1b7a95dd44dce, user_id=793cffd0e0c04069ba94846af1139f2d, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.349713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1e181-dc7e-4f12-99c2-36973ed9d27f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.351823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 804/1606] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.352842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be8112-6657-4753-ac09-13034678cba4 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.353122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be8112-6657-4753-ac09-13034678cba4 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.353395 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be8112-6657-4753-ac09-13034678cba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.383235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-55be8112-6657-4753-ac09-13034678cba4 None None] MFA Rules not processed for user `7127a5d0909a49b9b8ee4c472ac28012`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:04.384713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-55be8112-6657-4753-ac09-13034678cba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:04.394793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:04.459982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 805/1607] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:04.459982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be8112-6657-4753-ac09-13034678cba4 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.471217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be8112-6657-4753-ac09-13034678cba4 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.476381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587c397-1a59-4cb4-9a8a-c6c9fd5553e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.476718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587c397-1a59-4cb4-9a8a-c6c9fd5553e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.476917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587c397-1a59-4cb4-9a8a-c6c9fd5553e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.477240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587c397-1a59-4cb4-9a8a-c6c9fd5553e1 admin admin] PATH_INFO: `/v3/projects/3b8ed5fb092140a1a4e9ae005f3c8973/users/60dc4976d686449f80d3bfad387a2406/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.494125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587c397-1a59-4cb4-9a8a-c6c9fd5553e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b8ed5fb092140a1a4e9ae005f3c8973, user_id=60dc4976d686449f80d3bfad387a2406, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.495230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING py.warnings [None req-6587c397-1a59-4cb4-9a8a-c6c9fd5553e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 17:00:04.495230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: warnings.warn(msg) Aug 08 17:00:04.495230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: Aug 08 17:00:04.497214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587c397-1a59-4cb4-9a8a-c6c9fd5553e1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.515020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 803/1608] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:04 2022] PUT /identity/v3/projects/95d311c9e53e4b2f89a1b7a95dd44dce/users/793cffd0e0c04069ba94846af1139f2d/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:04.520156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1e181-dc7e-4f12-99c2-36973ed9d27f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.577422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1e181-dc7e-4f12-99c2-36973ed9d27f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.581142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4faf32-3fe5-4047-af9c-5f63dee7e4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.581767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4faf32-3fe5-4047-af9c-5f63dee7e4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.582069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4faf32-3fe5-4047-af9c-5f63dee7e4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.582339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4faf32-3fe5-4047-af9c-5f63dee7e4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.598960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 806/1609] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:04 2022] PUT /identity/v3/projects/3b8ed5fb092140a1a4e9ae005f3c8973/users/60dc4976d686449f80d3bfad387a2406/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:04.598960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587c397-1a59-4cb4-9a8a-c6c9fd5553e1 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.646116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4faf32-3fe5-4047-af9c-5f63dee7e4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.647641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4faf32-3fe5-4047-af9c-5f63dee7e4c3 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.657386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 804/1610] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:04.658024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4faf32-3fe5-4047-af9c-5f63dee7e4c3 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.659021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587c397-1a59-4cb4-9a8a-c6c9fd5553e1 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.665363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54f932-2564-4afa-9c44-3845e8dca83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.666191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54f932-2564-4afa-9c44-3845e8dca83f admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.666575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54f932-2564-4afa-9c44-3845e8dca83f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.666951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54f932-2564-4afa-9c44-3845e8dca83f admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.667660 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4faf32-3fe5-4047-af9c-5f63dee7e4c3 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.667908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54f932-2564-4afa-9c44-3845e8dca83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.669621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54f932-2564-4afa-9c44-3845e8dca83f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.672999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5c8e6-fef0-44be-8eed-564d316e0980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.673378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5c8e6-fef0-44be-8eed-564d316e0980 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.673652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5c8e6-fef0-44be-8eed-564d316e0980 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.674018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5c8e6-fef0-44be-8eed-564d316e0980 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.674726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5c8e6-fef0-44be-8eed-564d316e0980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.676218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5c8e6-fef0-44be-8eed-564d316e0980 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.681950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 807/1611] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.683111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54f932-2564-4afa-9c44-3845e8dca83f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.691667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 805/1612] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.691667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5c8e6-fef0-44be-8eed-564d316e0980 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.696171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54f932-2564-4afa-9c44-3845e8dca83f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.701118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43a36e-550d-4b2e-a429-721046fdb7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.701832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43a36e-550d-4b2e-a429-721046fdb7a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.702070 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5c8e6-fef0-44be-8eed-564d316e0980 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.702196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43a36e-550d-4b2e-a429-721046fdb7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.702490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43a36e-550d-4b2e-a429-721046fdb7a5 admin admin] PATH_INFO: `/v3/projects/1a3cc7413c0c4603a7898fd5b0d6a765/users/d6e9bd6db85b4decbdac1908912d158c/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.706915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7379f58-cbd9-4999-bf13-3bc15b85c39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.707588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7379f58-cbd9-4999-bf13-3bc15b85c39e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.707963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7379f58-cbd9-4999-bf13-3bc15b85c39e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.708325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7379f58-cbd9-4999-bf13-3bc15b85c39e admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.709094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7379f58-cbd9-4999-bf13-3bc15b85c39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.713889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7379f58-cbd9-4999-bf13-3bc15b85c39e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.718976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43a36e-550d-4b2e-a429-721046fdb7a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a3cc7413c0c4603a7898fd5b0d6a765, user_id=d6e9bd6db85b4decbdac1908912d158c, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.721334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 806/1613] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.721696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43a36e-550d-4b2e-a429-721046fdb7a5 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.722709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53515b38-57ee-4dbd-9078-0f9faaeb18db None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.723002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53515b38-57ee-4dbd-9078-0f9faaeb18db None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.723299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53515b38-57ee-4dbd-9078-0f9faaeb18db None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.724855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 807/1614] 10.222.0.57 () {54 vars in 962 bytes} [Mon Aug 8 17:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:00:04.725686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53515b38-57ee-4dbd-9078-0f9faaeb18db None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.744103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53515b38-57ee-4dbd-9078-0f9faaeb18db None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.748971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b1148-99a5-48be-a881-c71889c7b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.749611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b1148-99a5-48be-a881-c71889c7b7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.749959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b1148-99a5-48be-a881-c71889c7b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.750220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b1148-99a5-48be-a881-c71889c7b7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.750859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b1148-99a5-48be-a881-c71889c7b7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.752493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b1148-99a5-48be-a881-c71889c7b7d9 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.767165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 808/1615] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:04.767165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8215a4e-a135-4506-a535-8991070f5669 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.767165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8215a4e-a135-4506-a535-8991070f5669 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.767165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8215a4e-a135-4506-a535-8991070f5669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.786291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f8215a4e-a135-4506-a535-8991070f5669 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:04.787783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f8215a4e-a135-4506-a535-8991070f5669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:04.800314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 808/1616] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:04 2022] PUT /identity/v3/projects/1a3cc7413c0c4603a7898fd5b0d6a765/users/d6e9bd6db85b4decbdac1908912d158c/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:04.800314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43a36e-550d-4b2e-a429-721046fdb7a5 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.803328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:04.880750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 809/1617] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:04.881813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8215a4e-a135-4506-a535-8991070f5669 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:04.892327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8215a4e-a135-4506-a535-8991070f5669 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.898320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1dc8-9353-4d0a-b8e8-6b7c77fe87c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.899044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1dc8-9353-4d0a-b8e8-6b7c77fe87c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.899325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1dc8-9353-4d0a-b8e8-6b7c77fe87c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.899589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1dc8-9353-4d0a-b8e8-6b7c77fe87c6 admin admin] PATH_INFO: `/v3/projects/9a1e963209924abb967428435401540a/users/4bbd40052949468a897744a3010f337c/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.910898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43a36e-550d-4b2e-a429-721046fdb7a5 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:04.918069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1dc8-9353-4d0a-b8e8-6b7c77fe87c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1e963209924abb967428435401540a, user_id=4bbd40052949468a897744a3010f337c, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.921726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd94aac-3091-4485-a7a5-a8c5b6f706f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:04.921726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd94aac-3091-4485-a7a5-a8c5b6f706f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:04.921726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd94aac-3091-4485-a7a5-a8c5b6f706f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:04.921726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd94aac-3091-4485-a7a5-a8c5b6f706f2 admin admin] PATH_INFO: `/v3/projects/a9fca728184d4db3ba7db90d0a1c99ab/users/86b753bb6bf84b968ecba14b30f6bb9a/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:04.922272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1dc8-9353-4d0a-b8e8-6b7c77fe87c6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:04.922898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:04.937189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd94aac-3091-4485-a7a5-a8c5b6f706f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9fca728184d4db3ba7db90d0a1c99ab, user_id=86b753bb6bf84b968ecba14b30f6bb9a, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:04.940014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd94aac-3091-4485-a7a5-a8c5b6f706f2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:05.073153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 809/1618] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:04 2022] PUT /identity/v3/projects/a9fca728184d4db3ba7db90d0a1c99ab/users/86b753bb6bf84b968ecba14b30f6bb9a/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:05.075193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 810/1619] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:04 2022] PUT /identity/v3/projects/9a1e963209924abb967428435401540a/users/4bbd40052949468a897744a3010f337c/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:05.076271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e024ec-7f26-4d4c-8e74-684f7e559692 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.076632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e024ec-7f26-4d4c-8e74-684f7e559692 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.076979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e024ec-7f26-4d4c-8e74-684f7e559692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.084356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd94aac-3091-4485-a7a5-a8c5b6f706f2 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:05.132136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b3e024ec-7f26-4d4c-8e74-684f7e559692 None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:05.134195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b3e024ec-7f26-4d4c-8e74-684f7e559692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:05.180287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd94aac-3091-4485-a7a5-a8c5b6f706f2 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:05.185199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84503b-32ab-41bf-9439-0b76c179e7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:05.185933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84503b-32ab-41bf-9439-0b76c179e7d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.186212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84503b-32ab-41bf-9439-0b76c179e7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.186482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84503b-32ab-41bf-9439-0b76c179e7d0 admin admin] PATH_INFO: `/v3/projects/2a931bdc688c44559790e60e398714c9/users/905685acada94c67a2e495d6a2ffcec5/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.201767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84503b-32ab-41bf-9439-0b76c179e7d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a931bdc688c44559790e60e398714c9, user_id=905685acada94c67a2e495d6a2ffcec5, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:05.206515 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84503b-32ab-41bf-9439-0b76c179e7d0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:05.206773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 811/1620] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Aug 8 17:00:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:05.207240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e024ec-7f26-4d4c-8e74-684f7e559692 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:05.219152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e024ec-7f26-4d4c-8e74-684f7e559692 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:05.223476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aef0e1-fff0-4474-bbd8-82ef69344504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:05.224427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aef0e1-fff0-4474-bbd8-82ef69344504 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.224427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aef0e1-fff0-4474-bbd8-82ef69344504 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.224668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aef0e1-fff0-4474-bbd8-82ef69344504 admin admin] PATH_INFO: `/v3/projects/95d311c9e53e4b2f89a1b7a95dd44dce/users/793cffd0e0c04069ba94846af1139f2d/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.226627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aef0e1-fff0-4474-bbd8-82ef69344504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d311c9e53e4b2f89a1b7a95dd44dce, user_id=793cffd0e0c04069ba94846af1139f2d, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:05.229299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aef0e1-fff0-4474-bbd8-82ef69344504 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:05.239742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 812/1621] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:05 2022] PUT /identity/v3/projects/95d311c9e53e4b2f89a1b7a95dd44dce/users/793cffd0e0c04069ba94846af1139f2d/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:05.240997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37526ec3-d3d2-4f44-9593-ac9754c11c9b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.244501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37526ec3-d3d2-4f44-9593-ac9754c11c9b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.244501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37526ec3-d3d2-4f44-9593-ac9754c11c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.262367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-37526ec3-d3d2-4f44-9593-ac9754c11c9b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:05.263653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-37526ec3-d3d2-4f44-9593-ac9754c11c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:05.312419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:05.324532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 813/1622] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:05.325180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37526ec3-d3d2-4f44-9593-ac9754c11c9b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:05.326601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-37526ec3-d3d2-4f44-9593-ac9754c11c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:05.357767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 810/1623] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:05 2022] PUT /identity/v3/projects/2a931bdc688c44559790e60e398714c9/users/905685acada94c67a2e495d6a2ffcec5/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:05.361682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad3b6d-5352-4ce2-a140-6d662157fa05 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.361992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad3b6d-5352-4ce2-a140-6d662157fa05 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.362263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad3b6d-5352-4ce2-a140-6d662157fa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.394854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37526ec3-d3d2-4f44-9593-ac9754c11c9b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:05.402950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f1ad3b6d-5352-4ce2-a140-6d662157fa05 None None] MFA Rules not processed for user `4bbd40052949468a897744a3010f337c`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:05.408094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f1ad3b6d-5352-4ce2-a140-6d662157fa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:05.456654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9d7f-8375-4d20-8e7b-05cbd657fd91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:05.457490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9d7f-8375-4d20-8e7b-05cbd657fd91 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.457710 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9d7f-8375-4d20-8e7b-05cbd657fd91 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.457999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9d7f-8375-4d20-8e7b-05cbd657fd91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.484694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 811/1624] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:05.490590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec121d65-2788-4d66-89cc-d302f90f0c11 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.490877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec121d65-2788-4d66-89cc-d302f90f0c11 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.491144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec121d65-2788-4d66-89cc-d302f90f0c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.523898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ec121d65-2788-4d66-89cc-d302f90f0c11 None None] MFA Rules not processed for user `86b753bb6bf84b968ecba14b30f6bb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:05.525452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ec121d65-2788-4d66-89cc-d302f90f0c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:05.552701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b9d7f-8375-4d20-8e7b-05cbd657fd91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:05.555124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b9d7f-8375-4d20-8e7b-05cbd657fd91 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:05.569040 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 814/1625] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:05.570808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9d7f-8375-4d20-8e7b-05cbd657fd91 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:05.591901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:05.619877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 812/1626] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:05 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:05.621283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac0710-a674-4953-a14b-908b06bef7de None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.621636 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac0710-a674-4953-a14b-908b06bef7de None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.622009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac0710-a674-4953-a14b-908b06bef7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.648665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-42ac0710-a674-4953-a14b-908b06bef7de None None] MFA Rules not processed for user `793cffd0e0c04069ba94846af1139f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:05.649946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-42ac0710-a674-4953-a14b-908b06bef7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:05.650076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9d7f-8375-4d20-8e7b-05cbd657fd91 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:05.655847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669608f-3e72-461c-b409-19fd2a137809 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:05.656226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669608f-3e72-461c-b409-19fd2a137809 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.656509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669608f-3e72-461c-b409-19fd2a137809 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.656898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669608f-3e72-461c-b409-19fd2a137809 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.714565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669608f-3e72-461c-b409-19fd2a137809 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:05.716176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669608f-3e72-461c-b409-19fd2a137809 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:05.727602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 813/1627] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:05.727842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 815/1628] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:05.728752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2e88f-f6a1-4fcd-a3bf-0fd7c0fd1413 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.729047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2e88f-f6a1-4fcd-a3bf-0fd7c0fd1413 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.729303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2e88f-f6a1-4fcd-a3bf-0fd7c0fd1413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.731080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac0710-a674-4953-a14b-908b06bef7de None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:05.743786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac0710-a674-4953-a14b-908b06bef7de None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:05.747936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500dff4-15e8-4b4f-85d9-b3de9d71f506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:05.756162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500dff4-15e8-4b4f-85d9-b3de9d71f506 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.756162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500dff4-15e8-4b4f-85d9-b3de9d71f506 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.756162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500dff4-15e8-4b4f-85d9-b3de9d71f506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.759714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-45c2e88f-f6a1-4fcd-a3bf-0fd7c0fd1413 None None] MFA Rules not processed for user `905685acada94c67a2e495d6a2ffcec5`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:05.761162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-45c2e88f-f6a1-4fcd-a3bf-0fd7c0fd1413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:05.762008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500dff4-15e8-4b4f-85d9-b3de9d71f506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:05.763547 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500dff4-15e8-4b4f-85d9-b3de9d71f506 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:05.780149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 814/1629] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:05.781485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bf82b-1ce4-417d-b21b-fc6607edc65b None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.782200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bf82b-1ce4-417d-b21b-fc6607edc65b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.782200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bf82b-1ce4-417d-b21b-fc6607edc65b None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.783827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 815/1630] 10.222.0.57 () {54 vars in 962 bytes} [Mon Aug 8 17:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:00:05.784559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bf82b-1ce4-417d-b21b-fc6607edc65b None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:05.799272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bf82b-1ce4-417d-b21b-fc6607edc65b None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:05.804523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ded4-787e-4099-b793-4cc82eb567d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:05.805380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ded4-787e-4099-b793-4cc82eb567d1 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.806127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ded4-787e-4099-b793-4cc82eb567d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.806127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ded4-787e-4099-b793-4cc82eb567d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.806893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385ded4-787e-4099-b793-4cc82eb567d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:05.808903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385ded4-787e-4099-b793-4cc82eb567d1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:05.855071 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 816/1631] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:05.856306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2e88f-f6a1-4fcd-a3bf-0fd7c0fd1413 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:05.916012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2e88f-f6a1-4fcd-a3bf-0fd7c0fd1413 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:05.920150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e56c3-46e6-4c91-afde-040cc02e5603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:05.921092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e56c3-46e6-4c91-afde-040cc02e5603 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.921092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e56c3-46e6-4c91-afde-040cc02e5603 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.921400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e56c3-46e6-4c91-afde-040cc02e5603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.933335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e56c3-46e6-4c91-afde-040cc02e5603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:05.935568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e56c3-46e6-4c91-afde-040cc02e5603 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:05.950429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 817/1632] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:05.951900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0f51a-2c8d-4228-8641-33076369cd6d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:05.952297 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0f51a-2c8d-4228-8641-33076369cd6d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:05.952655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0f51a-2c8d-4228-8641-33076369cd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:05.975114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9bd0f51a-2c8d-4228-8641-33076369cd6d None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:05.977132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9bd0f51a-2c8d-4228-8641-33076369cd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:05.982109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 816/1633] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:05 2022] POST /identity/v3/projects => generated 403 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:05.982496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ded4-787e-4099-b793-4cc82eb567d1 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.049291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ded4-787e-4099-b793-4cc82eb567d1 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:06.054909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ab049-4414-4d11-bccc-58396ffe88fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:06.060905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 818/1634] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Aug 8 17:00:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:06.060905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41879f-76bf-4050-bf75-b375947012c9 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.060905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41879f-76bf-4050-bf75-b375947012c9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.060905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41879f-76bf-4050-bf75-b375947012c9 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.061416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ab049-4414-4d11-bccc-58396ffe88fd admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.061528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 819/1635] 10.222.0.57 () {54 vars in 912 bytes} [Mon Aug 8 17:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:00:06.061528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41879f-76bf-4050-bf75-b375947012c9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.064874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ab049-4414-4d11-bccc-58396ffe88fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.065330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ab049-4414-4d11-bccc-58396ffe88fd admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.066174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ab049-4414-4d11-bccc-58396ffe88fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:06.068351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ab049-4414-4d11-bccc-58396ffe88fd admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:06.075156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41879f-76bf-4050-bf75-b375947012c9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:06.091056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f40fc-744f-4ab1-9902-a4dc4a1df42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:06.092021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f40fc-744f-4ab1-9902-a4dc4a1df42e admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.096350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f40fc-744f-4ab1-9902-a4dc4a1df42e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.096350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f40fc-744f-4ab1-9902-a4dc4a1df42e admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.096350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f40fc-744f-4ab1-9902-a4dc4a1df42e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:06.096350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f40fc-744f-4ab1-9902-a4dc4a1df42e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:06.191480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 820/1636] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:06 2022] POST /identity/v3/users => generated 308 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:06.195634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 817/1637] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:05 2022] POST /identity/v3/projects => generated 371 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:06.195634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915c535-f9f2-4bd0-a618-dce892de6048 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.195634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915c535-f9f2-4bd0-a618-dce892de6048 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.195634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915c535-f9f2-4bd0-a618-dce892de6048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.196933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f40fc-744f-4ab1-9902-a4dc4a1df42e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.210470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f40fc-744f-4ab1-9902-a4dc4a1df42e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:06.216767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d915c535-f9f2-4bd0-a618-dce892de6048 None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:06.218338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d915c535-f9f2-4bd0-a618-dce892de6048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:06.301981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c373c-5ac5-4c07-a91e-7137e7b84fbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:06.302701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c373c-5ac5-4c07-a91e-7137e7b84fbf service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.303076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c373c-5ac5-4c07-a91e-7137e7b84fbf service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.303455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c373c-5ac5-4c07-a91e-7137e7b84fbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.311469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 818/1638] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:00:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:06.320044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915c535-f9f2-4bd0-a618-dce892de6048 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.364207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c373c-5ac5-4c07-a91e-7137e7b84fbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:06.365966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c373c-5ac5-4c07-a91e-7137e7b84fbf service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:06.374904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 821/1639] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:06.375737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c373c-5ac5-4c07-a91e-7137e7b84fbf service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.386704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c373c-5ac5-4c07-a91e-7137e7b84fbf service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:06.389010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915c535-f9f2-4bd0-a618-dce892de6048 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:06.394521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f42f-36e6-498d-a4da-4a2ad3f80e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:06.395740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f42f-36e6-498d-a4da-4a2ad3f80e1d admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.396025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f42f-36e6-498d-a4da-4a2ad3f80e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.396326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f42f-36e6-498d-a4da-4a2ad3f80e1d admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.396961 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81f42f-36e6-498d-a4da-4a2ad3f80e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:06.399452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81f42f-36e6-498d-a4da-4a2ad3f80e1d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:06.403654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a2b8f-b93b-4a19-a069-90aa3aee365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:06.404269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2b8f-b93b-4a19-a069-90aa3aee365a admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.404542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2b8f-b93b-4a19-a069-90aa3aee365a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.404834 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2b8f-b93b-4a19-a069-90aa3aee365a admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.405499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a2b8f-b93b-4a19-a069-90aa3aee365a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:06.407116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a2b8f-b93b-4a19-a069-90aa3aee365a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:06.418827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 819/1640] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:06.420002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a2b8f-b93b-4a19-a069-90aa3aee365a admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.428038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:06.484078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a2b8f-b93b-4a19-a069-90aa3aee365a admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:06.489073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b4a4f-bebd-4e20-bd99-39962a0081fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:06.489751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b4a4f-bebd-4e20-bd99-39962a0081fc service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.490086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b4a4f-bebd-4e20-bd99-39962a0081fc service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.490371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b4a4f-bebd-4e20-bd99-39962a0081fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.509665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b4a4f-bebd-4e20-bd99-39962a0081fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:06.509665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b4a4f-bebd-4e20-bd99-39962a0081fc service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:06.523158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 822/1641] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:06 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:06.523495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 820/1642] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:06.525825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f42f-36e6-498d-a4da-4a2ad3f80e1d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.535414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b4a4f-bebd-4e20-bd99-39962a0081fc service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.560025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b4a4f-bebd-4e20-bd99-39962a0081fc service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:06.572560 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30440-8f5a-49ae-9b60-8fc064259996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:06.585529 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f42f-36e6-498d-a4da-4a2ad3f80e1d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:06.586178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30440-8f5a-49ae-9b60-8fc064259996 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.587922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30440-8f5a-49ae-9b60-8fc064259996 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.588183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30440-8f5a-49ae-9b60-8fc064259996 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.592094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30440-8f5a-49ae-9b60-8fc064259996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:06.594038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfa4d9-53c4-406f-adcf-305fcf34e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:06.595171 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfa4d9-53c4-406f-adcf-305fcf34e071 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.595752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30440-8f5a-49ae-9b60-8fc064259996 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:06.596191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfa4d9-53c4-406f-adcf-305fcf34e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.597840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfa4d9-53c4-406f-adcf-305fcf34e071 admin admin] PATH_INFO: `/v3/projects/771289073d714c189cab72f07957166b/users/c8d45ce5c95b4be899f1788aec069e4b/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.626411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 821/1643] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:06.630859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30440-8f5a-49ae-9b60-8fc064259996 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.648891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfa4d9-53c4-406f-adcf-305fcf34e071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771289073d714c189cab72f07957166b, user_id=c8d45ce5c95b4be899f1788aec069e4b, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:06.650155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfa4d9-53c4-406f-adcf-305fcf34e071 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:06.655760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30440-8f5a-49ae-9b60-8fc064259996 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:06.664099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea43f04-f76f-4036-a088-6d6a998bcf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:06.664782 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea43f04-f76f-4036-a088-6d6a998bcf33 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.665059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea43f04-f76f-4036-a088-6d6a998bcf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.665333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea43f04-f76f-4036-a088-6d6a998bcf33 admin admin] PATH_INFO: `/v3/projects/16c7db134ed345429ef0bdb3261328de/users/9f479dea0cac41fd8086a2ebc42434ed/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.677960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:06.687272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea43f04-f76f-4036-a088-6d6a998bcf33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c7db134ed345429ef0bdb3261328de, user_id=9f479dea0cac41fd8086a2ebc42434ed, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:06.690248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea43f04-f76f-4036-a088-6d6a998bcf33 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:06.764350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 823/1644] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:06 2022] PUT /identity/v3/projects/771289073d714c189cab72f07957166b/users/c8d45ce5c95b4be899f1788aec069e4b/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:06.769959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaa9c5-d9ac-4663-b76f-ade26ff39999 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.769959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaa9c5-d9ac-4663-b76f-ade26ff39999 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.769959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaa9c5-d9ac-4663-b76f-ade26ff39999 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.772894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 824/1645] 10.222.0.57 () {54 vars in 912 bytes} [Mon Aug 8 17:00:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:00:06.773778 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaa9c5-d9ac-4663-b76f-ade26ff39999 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:06.805718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 822/1646] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:06 2022] PUT /identity/v3/projects/16c7db134ed345429ef0bdb3261328de/users/9f479dea0cac41fd8086a2ebc42434ed/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:06.810288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b91d1-2e00-4744-823b-cc7c380d52ce None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:06.810288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b91d1-2e00-4744-823b-cc7c380d52ce None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:06.810288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b91d1-2e00-4744-823b-cc7c380d52ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:06.844213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-537b91d1-2e00-4744-823b-cc7c380d52ce None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:06.844522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:06.847734 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-537b91d1-2e00-4744-823b-cc7c380d52ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:06.907460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaa9c5-d9ac-4663-b76f-ade26ff39999 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:07.019655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3066c-9fb3-4885-82b0-97778e0b3e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:07.020307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3066c-9fb3-4885-82b0-97778e0b3e46 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:07.020582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3066c-9fb3-4885-82b0-97778e0b3e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:07.020902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3066c-9fb3-4885-82b0-97778e0b3e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:07.021636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3066c-9fb3-4885-82b0-97778e0b3e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:07.022242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 823/1647] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:00:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:07.023276 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3066c-9fb3-4885-82b0-97778e0b3e46 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:07.024083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b91d1-2e00-4744-823b-cc7c380d52ce None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:07.041525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 825/1648] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:07.047918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b91d1-2e00-4744-823b-cc7c380d52ce None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:07.048085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3066c-9fb3-4885-82b0-97778e0b3e46 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:07.057531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3066c-9fb3-4885-82b0-97778e0b3e46 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:07.062305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8841d-d7fa-498e-8801-328e2ca08593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:07.063075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8841d-d7fa-498e-8801-328e2ca08593 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:07.063458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8841d-d7fa-498e-8801-328e2ca08593 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:07.063852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8841d-d7fa-498e-8801-328e2ca08593 admin admin] PATH_INFO: `/v3/projects/771289073d714c189cab72f07957166b/users/c8d45ce5c95b4be899f1788aec069e4b/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:07.066259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8841d-d7fa-498e-8801-328e2ca08593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771289073d714c189cab72f07957166b, user_id=c8d45ce5c95b4be899f1788aec069e4b, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:07.068880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d784f-5422-486c-a7c8-71869b76fb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:07.069650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8841d-d7fa-498e-8801-328e2ca08593 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:07.070367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d784f-5422-486c-a7c8-71869b76fb4d admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:07.071132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d784f-5422-486c-a7c8-71869b76fb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:07.072209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d784f-5422-486c-a7c8-71869b76fb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:07.073242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d784f-5422-486c-a7c8-71869b76fb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:07.075837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d784f-5422-486c-a7c8-71869b76fb4d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:07.088142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 826/1649] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:07 2022] PUT /identity/v3/projects/771289073d714c189cab72f07957166b/users/c8d45ce5c95b4be899f1788aec069e4b/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:07.088142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8841d-d7fa-498e-8801-328e2ca08593 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:07.090075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 824/1650] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:07.093858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d784f-5422-486c-a7c8-71869b76fb4d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:07.105711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d784f-5422-486c-a7c8-71869b76fb4d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:07.165067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8841d-d7fa-498e-8801-328e2ca08593 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:07.173049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa23b4-1663-4728-bdb9-e08d6d9e48d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:07.173677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa23b4-1663-4728-bdb9-e08d6d9e48d9 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:07.174186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa23b4-1663-4728-bdb9-e08d6d9e48d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:07.174584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa23b4-1663-4728-bdb9-e08d6d9e48d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:07.176559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc08d0-8f68-4d68-9dae-08bf6df6f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:07.177446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc08d0-8f68-4d68-9dae-08bf6df6f248 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:07.178373 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc08d0-8f68-4d68-9dae-08bf6df6f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:07.178760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc08d0-8f68-4d68-9dae-08bf6df6f248 admin admin] PATH_INFO: `/v3/projects/16c7db134ed345429ef0bdb3261328de/users/9f479dea0cac41fd8086a2ebc42434ed/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:07.181744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc08d0-8f68-4d68-9dae-08bf6df6f248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c7db134ed345429ef0bdb3261328de, user_id=9f479dea0cac41fd8086a2ebc42434ed, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:07.185825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc08d0-8f68-4d68-9dae-08bf6df6f248 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:07.207793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 825/1651] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:07 2022] PUT /identity/v3/projects/16c7db134ed345429ef0bdb3261328de/users/9f479dea0cac41fd8086a2ebc42434ed/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:07.214391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ecce0-19c0-4ce7-9712-e68db07850ea None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:07.214391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ecce0-19c0-4ce7-9712-e68db07850ea None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:07.214391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ecce0-19c0-4ce7-9712-e68db07850ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:07.251099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-840ecce0-19c0-4ce7-9712-e68db07850ea None None] MFA Rules not processed for user `c8d45ce5c95b4be899f1788aec069e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:07.252579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-840ecce0-19c0-4ce7-9712-e68db07850ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:07.261866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa23b4-1663-4728-bdb9-e08d6d9e48d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:07.264536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa23b4-1663-4728-bdb9-e08d6d9e48d9 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:07.294656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 827/1652] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:07.296527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89c8d-f18f-4abe-b116-9a2e5488cc1e None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:07.298534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89c8d-f18f-4abe-b116-9a2e5488cc1e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:07.299337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89c8d-f18f-4abe-b116-9a2e5488cc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:07.364124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-63f89c8d-f18f-4abe-b116-9a2e5488cc1e None None] MFA Rules not processed for user `9f479dea0cac41fd8086a2ebc42434ed`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:07.366964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-63f89c8d-f18f-4abe-b116-9a2e5488cc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:07.376826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 826/1653] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:07 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:07.407447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:07.470184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 828/1654] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:08.006776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:08.474096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ecce0-19c0-4ce7-9712-e68db07850ea None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:08.629164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ecce0-19c0-4ce7-9712-e68db07850ea None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:08.633725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387673b-2114-4875-b960-a849ad3266a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:08.635544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387673b-2114-4875-b960-a849ad3266a1 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:08.636403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387673b-2114-4875-b960-a849ad3266a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:08.641478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387673b-2114-4875-b960-a849ad3266a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:08.649492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387673b-2114-4875-b960-a849ad3266a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:08.651796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387673b-2114-4875-b960-a849ad3266a1 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:08.665215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 827/1655] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:08 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:08.737108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f89c8d-f18f-4abe-b116-9a2e5488cc1e None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:08.749009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f89c8d-f18f-4abe-b116-9a2e5488cc1e None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:08.765020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f8a98-00e8-467a-ab0b-07fc779487f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:08.765666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f8a98-00e8-467a-ab0b-07fc779487f4 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:08.765974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f8a98-00e8-467a-ab0b-07fc779487f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:08.766247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f8a98-00e8-467a-ab0b-07fc779487f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:08.784784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f8a98-00e8-467a-ab0b-07fc779487f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:08.786130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f8a98-00e8-467a-ab0b-07fc779487f4 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:08.798675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 829/1656] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:00:09.311137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:09.539229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387673b-2114-4875-b960-a849ad3266a1 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:09.655405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387673b-2114-4875-b960-a849ad3266a1 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:09.660057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:09.661246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:09.661815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:09.662490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] PATH_INFO: `/v3/users/86b753bb6bf84b968ecba14b30f6bb9a` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:09.665181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b753bb6bf84b968ecba14b30f6bb9a)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:09.666489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: WARNING py.warnings [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 17:00:09.666489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: warnings.warn(msg) Aug 08 17:00:09.666489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: Aug 08 17:00:09.668577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:10.107556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b753bb6bf84b968ecba14b30f6bb9a'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:10.285542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b753bb6bf84b968ecba14b30f6bb9a'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:10.298791 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b753bb6bf84b968ecba14b30f6bb9a'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:10.352178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b753bb6bf84b968ecba14b30f6bb9a'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:10.352178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b753bb6bf84b968ecba14b30f6bb9a'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:10.352178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 828/1657] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:09 2022] DELETE /identity/v3/users/86b753bb6bf84b968ecba14b30f6bb9a => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:10.356748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c790b3d-ed03-46e0-bdb7-c2e34dce430c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:10.357405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c790b3d-ed03-46e0-bdb7-c2e34dce430c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:10.357405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c790b3d-ed03-46e0-bdb7-c2e34dce430c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:10.374108 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:10.392736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-6c790b3d-ed03-46e0-bdb7-c2e34dce430c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:10.394728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6c790b3d-ed03-46e0-bdb7-c2e34dce430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:10.603776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 830/1658] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:10.649546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:10.654208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:10.760153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8d18b-6c43-4d41-b6a7-a4ac6419f571 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:10.764015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485754ae-b1b0-4747-8b0e-6fa2ea53ce93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:10.764781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485754ae-b1b0-4747-8b0e-6fa2ea53ce93 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:10.765249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485754ae-b1b0-4747-8b0e-6fa2ea53ce93 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:10.765669 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485754ae-b1b0-4747-8b0e-6fa2ea53ce93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:10.782247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485754ae-b1b0-4747-8b0e-6fa2ea53ce93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:10.783382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485754ae-b1b0-4747-8b0e-6fa2ea53ce93 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:10.796779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 829/1659] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:11.980753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c790b3d-ed03-46e0-bdb7-c2e34dce430c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:11.982683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6c790b3d-ed03-46e0-bdb7-c2e34dce430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:11.999161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c790b3d-ed03-46e0-bdb7-c2e34dce430c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:12.005943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:12.007304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:12.007848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:12.008324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] PATH_INFO: `/v3/users/905685acada94c67a2e495d6a2ffcec5` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:12.010038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=905685acada94c67a2e495d6a2ffcec5)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:12.012687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:12.373957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485754ae-b1b0-4747-8b0e-6fa2ea53ce93 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:12.375575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-485754ae-b1b0-4747-8b0e-6fa2ea53ce93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:12.402533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485754ae-b1b0-4747-8b0e-6fa2ea53ce93 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:12.410835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:12.412536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:12.412977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:12.413567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] PATH_INFO: `/v3/projects/a9fca728184d4db3ba7db90d0a1c99ab` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:12.415631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9fca728184d4db3ba7db90d0a1c99ab)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:12.418990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:12.555747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:12.579829 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905685acada94c67a2e495d6a2ffcec5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.696939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905685acada94c67a2e495d6a2ffcec5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.720639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905685acada94c67a2e495d6a2ffcec5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.721487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:12.742885 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905685acada94c67a2e495d6a2ffcec5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.743659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a782a873-c52d-4e96-8012-72d4b8943796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905685acada94c67a2e495d6a2ffcec5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.753021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 831/1660] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:11 2022] DELETE /identity/v3/users/905685acada94c67a2e495d6a2ffcec5 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:12.754639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f87da1-ee79-4096-9d5d-a2e5f231253b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:12.755087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f87da1-ee79-4096-9d5d-a2e5f231253b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:12.755523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f87da1-ee79-4096-9d5d-a2e5f231253b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:12.760222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9fca728184d4db3ba7db90d0a1c99ab', 'request_id': 'req-b15384e1-5067-4db2-8465-485700f3e3f9'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.786725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9fca728184d4db3ba7db90d0a1c99ab', 'request_id': 'req-b15384e1-5067-4db2-8465-485700f3e3f9'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.810027 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-10f87da1-ee79-4096-9d5d-a2e5f231253b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:12.811234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-10f87da1-ee79-4096-9d5d-a2e5f231253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:12.892946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9fca728184d4db3ba7db90d0a1c99ab', 'request_id': 'req-b15384e1-5067-4db2-8465-485700f3e3f9'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.911406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] The token cache is being invalidate because project a9fca728184d4db3ba7db90d0a1c99ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:12.911820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.913085 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:12.916584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 832/1662] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:12.919226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 830/1662] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:12 2022] DELETE /identity/v3/projects/a9fca728184d4db3ba7db90d0a1c99ab => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:12.921109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f87da1-ee79-4096-9d5d-a2e5f231253b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:12.923161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-10f87da1-ee79-4096-9d5d-a2e5f231253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:12.927220 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:12.929085 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:12.954568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f87da1-ee79-4096-9d5d-a2e5f231253b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:12.962801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:12.964389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:12.964820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:12.965281 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] PATH_INFO: `/v3/users/c8d45ce5c95b4be899f1788aec069e4b` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:12.966752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d45ce5c95b4be899f1788aec069e4b)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:12.969749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:13.020820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15384e1-5067-4db2-8465-485700f3e3f9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:13.121906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:13.122845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:13.123128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:13.129101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:13.138015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:13.174027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:13.175568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:13.190661 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 831/1663] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:13.302926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d45ce5c95b4be899f1788aec069e4b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:13.475396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d45ce5c95b4be899f1788aec069e4b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:13.495751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d45ce5c95b4be899f1788aec069e4b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:13.512717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d45ce5c95b4be899f1788aec069e4b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:13.513614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d45ce5c95b4be899f1788aec069e4b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:13.522546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 833/1664] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:12 2022] DELETE /identity/v3/users/c8d45ce5c95b4be899f1788aec069e4b => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:14.315110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:14.316718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:14.348723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955b71e-c26e-46e5-a120-b05b49feaf91 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:14.416836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:14.418416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:14.573858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:14.574852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:14.575280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:14.576128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] PATH_INFO: `/v3/projects/771289073d714c189cab72f07957166b` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:14.578400 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] RBAC: Authorizing `identity:delete_project(project_id=771289073d714c189cab72f07957166b)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:14.592975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:14.652589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ac96c-72a0-4241-b433-d143e4f44c6f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:14.670652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:14.671645 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:14.675869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:14.676375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] PATH_INFO: `/v3/users/908b1d1f3c9f42e699680b98e3e4b360` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:14.678003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=908b1d1f3c9f42e699680b98e3e4b360)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:14.683041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:14.696099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:14.905131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771289073d714c189cab72f07957166b', 'request_id': 'req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:14.929193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771289073d714c189cab72f07957166b', 'request_id': 'req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.010741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771289073d714c189cab72f07957166b', 'request_id': 'req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.039256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] The token cache is being invalidate because project 771289073d714c189cab72f07957166b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:15.039929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.041007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d7d2697-2146-47c2-8b31-8fe3c77ce90e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.043156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 832/1665] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:14 2022] DELETE /identity/v3/projects/771289073d714c189cab72f07957166b => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:15.047011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908b1d1f3c9f42e699680b98e3e4b360'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.049462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b5005-2ee1-420a-8ea9-f3705459f402 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.050136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b5005-2ee1-420a-8ea9-f3705459f402 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.051393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b5005-2ee1-420a-8ea9-f3705459f402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.093495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5d8b5005-2ee1-420a-8ea9-f3705459f402 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:15.098965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5d8b5005-2ee1-420a-8ea9-f3705459f402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:15.144583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908b1d1f3c9f42e699680b98e3e4b360'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.164661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908b1d1f3c9f42e699680b98e3e4b360'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.184170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908b1d1f3c9f42e699680b98e3e4b360'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.184816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-948dd421-9d79-42ab-9537-5a19d618cba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908b1d1f3c9f42e699680b98e3e4b360'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.202663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 834/1666] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:14 2022] DELETE /identity/v3/users/908b1d1f3c9f42e699680b98e3e4b360 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:15.212381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 833/1667] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:15.214876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51c54f-f13f-4a29-aefa-e2d327638852 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.215160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51c54f-f13f-4a29-aefa-e2d327638852 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.216715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51c54f-f13f-4a29-aefa-e2d327638852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.216892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5782a2a-b5c3-4b1f-abb3-6a2fc8dd2e57 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.217329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5782a2a-b5c3-4b1f-abb3-6a2fc8dd2e57 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.217750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5782a2a-b5c3-4b1f-abb3-6a2fc8dd2e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.259694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a5782a2a-b5c3-4b1f-abb3-6a2fc8dd2e57 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:15.261682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4f51c54f-f13f-4a29-aefa-e2d327638852 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:15.262919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4f51c54f-f13f-4a29-aefa-e2d327638852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:15.263419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a5782a2a-b5c3-4b1f-abb3-6a2fc8dd2e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:15.369831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 835/1668] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:15.372594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5782a2a-b5c3-4b1f-abb3-6a2fc8dd2e57 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:15.397916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 834/1669] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:15.414109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51c54f-f13f-4a29-aefa-e2d327638852 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:15.416622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4f51c54f-f13f-4a29-aefa-e2d327638852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:15.421651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5782a2a-b5c3-4b1f-abb3-6a2fc8dd2e57 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:15.430488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971be82-0b87-4f9a-b1d5-893a8f867b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:15.431522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971be82-0b87-4f9a-b1d5-893a8f867b5b admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.431980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971be82-0b87-4f9a-b1d5-893a8f867b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.432587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971be82-0b87-4f9a-b1d5-893a8f867b5b admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.433445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971be82-0b87-4f9a-b1d5-893a8f867b5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:15.433966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971be82-0b87-4f9a-b1d5-893a8f867b5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:15.436374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971be82-0b87-4f9a-b1d5-893a8f867b5b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:15.459693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 836/1670] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:15.462479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971be82-0b87-4f9a-b1d5-893a8f867b5b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:15.482269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971be82-0b87-4f9a-b1d5-893a8f867b5b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:15.489101 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c917e-f5d0-4fec-8808-23a6a2e33922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:15.493149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c917e-f5d0-4fec-8808-23a6a2e33922 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.493149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c917e-f5d0-4fec-8808-23a6a2e33922 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.493149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c917e-f5d0-4fec-8808-23a6a2e33922 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.494012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c917e-f5d0-4fec-8808-23a6a2e33922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:15.497696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c917e-f5d0-4fec-8808-23a6a2e33922 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:15.528869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51c54f-f13f-4a29-aefa-e2d327638852 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:15.537537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a15f5-7b8b-45dc-a1f6-6ecad5685033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:15.539025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a15f5-7b8b-45dc-a1f6-6ecad5685033 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.539378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a15f5-7b8b-45dc-a1f6-6ecad5685033 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.539751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a15f5-7b8b-45dc-a1f6-6ecad5685033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.574429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a15f5-7b8b-45dc-a1f6-6ecad5685033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:15.577629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a15f5-7b8b-45dc-a1f6-6ecad5685033 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:15.605681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 835/1671] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:00:15.606936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a15f5-7b8b-45dc-a1f6-6ecad5685033 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:15.613135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4b4a15f5-7b8b-45dc-a1f6-6ecad5685033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:15.641509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a15f5-7b8b-45dc-a1f6-6ecad5685033 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:15.656824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:15.657721 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:15.658383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:15.658796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] PATH_INFO: `/v3/projects/2a931bdc688c44559790e60e398714c9` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:15.659834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 837/1672] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:15 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:15.660552 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a931bdc688c44559790e60e398714c9)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:15.662743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:15.664539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c917e-f5d0-4fec-8808-23a6a2e33922 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:15.860455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c917e-f5d0-4fec-8808-23a6a2e33922 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:15.898461 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a931bdc688c44559790e60e398714c9', 'request_id': 'req-16f70b8e-634e-4857-9ca0-a5fae3211831'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:15.926228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a931bdc688c44559790e60e398714c9', 'request_id': 'req-16f70b8e-634e-4857-9ca0-a5fae3211831'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.055280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:16.055540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317572cf-aad7-411e-a4cc-91ca00e0b749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:16.056144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317572cf-aad7-411e-a4cc-91ca00e0b749 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:16.056431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317572cf-aad7-411e-a4cc-91ca00e0b749 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:16.058144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a931bdc688c44559790e60e398714c9', 'request_id': 'req-16f70b8e-634e-4857-9ca0-a5fae3211831'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.058913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317572cf-aad7-411e-a4cc-91ca00e0b749 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:16.059871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317572cf-aad7-411e-a4cc-91ca00e0b749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:16.067011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317572cf-aad7-411e-a4cc-91ca00e0b749 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:16.084136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] The token cache is being invalidate because project 2a931bdc688c44559790e60e398714c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:16.084434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.086853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.093622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 836/1673] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:15 2022] DELETE /identity/v3/projects/2a931bdc688c44559790e60e398714c9 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:16.098133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:16.111146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:16.137525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f70b8e-634e-4857-9ca0-a5fae3211831 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:16.164770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:16.167226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:16.167827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:16.168399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] PATH_INFO: `/v3/users/9f479dea0cac41fd8086a2ebc42434ed` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:16.171147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f479dea0cac41fd8086a2ebc42434ed)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:16.175605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:16.217023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 838/1674] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:15 2022] POST /identity/v3/users => generated 290 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:16.219933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317572cf-aad7-411e-a4cc-91ca00e0b749 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:16.228275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-317572cf-aad7-411e-a4cc-91ca00e0b749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:16.256060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317572cf-aad7-411e-a4cc-91ca00e0b749 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:16.265806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd885391-e1c1-4d42-aef2-e85c364bac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:16.266929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd885391-e1c1-4d42-aef2-e85c364bac74 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:16.267109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd885391-e1c1-4d42-aef2-e85c364bac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:16.267596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd885391-e1c1-4d42-aef2-e85c364bac74 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:16.269549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd885391-e1c1-4d42-aef2-e85c364bac74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:16.276915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd885391-e1c1-4d42-aef2-e85c364bac74 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:16.301045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 839/1675] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:16.305517 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd885391-e1c1-4d42-aef2-e85c364bac74 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:16.329895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd885391-e1c1-4d42-aef2-e85c364bac74 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:16.340593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe2441-7e67-4a3e-a3c8-c72a0cee2499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:16.341273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe2441-7e67-4a3e-a3c8-c72a0cee2499 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:16.341557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe2441-7e67-4a3e-a3c8-c72a0cee2499 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:16.341850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe2441-7e67-4a3e-a3c8-c72a0cee2499 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:16.342520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe2441-7e67-4a3e-a3c8-c72a0cee2499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:16.344159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe2441-7e67-4a3e-a3c8-c72a0cee2499 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:16.368810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 840/1676] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:16.372630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe2441-7e67-4a3e-a3c8-c72a0cee2499 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:16.395399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe2441-7e67-4a3e-a3c8-c72a0cee2499 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:16.409427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1310011-ccd1-4154-80d8-d3f52318b5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:16.410141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1310011-ccd1-4154-80d8-d3f52318b5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:16.410420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1310011-ccd1-4154-80d8-d3f52318b5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:16.410692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1310011-ccd1-4154-80d8-d3f52318b5a7 admin admin] PATH_INFO: `/v3/projects/3bef8cad76964c0f844eea3fdd05eff1/users/158fac32de4148279458af2ab4a7ed85/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:16.435140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1310011-ccd1-4154-80d8-d3f52318b5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bef8cad76964c0f844eea3fdd05eff1, user_id=158fac32de4148279458af2ab4a7ed85, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:16.441556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1310011-ccd1-4154-80d8-d3f52318b5a7 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:16.507259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f479dea0cac41fd8086a2ebc42434ed'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.566244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 841/1677] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:16 2022] PUT /identity/v3/projects/3bef8cad76964c0f844eea3fdd05eff1/users/158fac32de4148279458af2ab4a7ed85/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:16.579339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3747b7-3b3e-492e-b37d-6c497f1a08a5 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:16.579625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3747b7-3b3e-492e-b37d-6c497f1a08a5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:16.579924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3747b7-3b3e-492e-b37d-6c497f1a08a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:16.610604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f479dea0cac41fd8086a2ebc42434ed'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.620872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f479dea0cac41fd8086a2ebc42434ed'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.662988 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f479dea0cac41fd8086a2ebc42434ed'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.666847 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f479dea0cac41fd8086a2ebc42434ed'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:16.686533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 837/1678] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:16 2022] DELETE /identity/v3/users/9f479dea0cac41fd8086a2ebc42434ed => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:16.692029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1e3747b7-3b3e-492e-b37d-6c497f1a08a5 None None] MFA Rules not processed for user `158fac32de4148279458af2ab4a7ed85`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:16.693295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1e3747b7-3b3e-492e-b37d-6c497f1a08a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:16.798603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 842/1679] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:16.803563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:16.803563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:16.894078 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3747b7-3b3e-492e-b37d-6c497f1a08a5 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:16.895663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1e3747b7-3b3e-492e-b37d-6c497f1a08a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:16.951524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c6f99-48af-4834-8743-4c400613635a admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:16.963356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2c9d-79c0-4af4-a8b2-9826a73d659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:16.967192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2c9d-79c0-4af4-a8b2-9826a73d659b admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:16.969543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2c9d-79c0-4af4-a8b2-9826a73d659b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:16.974913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2c9d-79c0-4af4-a8b2-9826a73d659b admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:16.974913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc2c9d-79c0-4af4-a8b2-9826a73d659b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:16.988528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc2c9d-79c0-4af4-a8b2-9826a73d659b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:17.041987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3747b7-3b3e-492e-b37d-6c497f1a08a5 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:17.063168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:17.063975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:17.064444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:17.064714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] PATH_INFO: `/v3/projects/8bc70ef2b50441e1baf52e05d005fe9a` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:17.066780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc70ef2b50441e1baf52e05d005fe9a)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:17.068029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: WARNING py.warnings [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 17:00:17.068029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: warnings.warn(msg) Aug 08 17:00:17.068029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: Aug 08 17:00:17.069715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:17.139008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 838/1680] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:16 2022] POST /identity/v3/projects => generated 381 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:17.142137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2c9d-79c0-4af4-a8b2-9826a73d659b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:17.209212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:17.359714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc70ef2b50441e1baf52e05d005fe9a', 'request_id': 'req-b7dfc631-380a-4cac-8188-58e1f425b5ad'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.419697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc70ef2b50441e1baf52e05d005fe9a', 'request_id': 'req-b7dfc631-380a-4cac-8188-58e1f425b5ad'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.425160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2c9d-79c0-4af4-a8b2-9826a73d659b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:17.520334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc70ef2b50441e1baf52e05d005fe9a', 'request_id': 'req-b7dfc631-380a-4cac-8188-58e1f425b5ad'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.545919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] The token cache is being invalidate because project 8bc70ef2b50441e1baf52e05d005fe9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:17.548491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.553109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:17.555613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 843/1681] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:16 2022] DELETE /identity/v3/projects/8bc70ef2b50441e1baf52e05d005fe9a => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:17.561180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:17.572779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:17.639060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c930c2-cd96-47a6-a006-ea9929664d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:17.639060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c930c2-cd96-47a6-a006-ea9929664d6d admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:17.644126 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c930c2-cd96-47a6-a006-ea9929664d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:17.645507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c930c2-cd96-47a6-a006-ea9929664d6d admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:17.648448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c930c2-cd96-47a6-a006-ea9929664d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:17.652048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c930c2-cd96-47a6-a006-ea9929664d6d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:17.758400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfc631-380a-4cac-8188-58e1f425b5ad admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:17.769918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 839/1682] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:17 2022] POST /identity/v3/users => generated 297 bytes in 629 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:17.771662 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f768d96-6bb2-4ebd-8e89-23c701e1b9bf None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:17.772409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f768d96-6bb2-4ebd-8e89-23c701e1b9bf None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:17.772533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f768d96-6bb2-4ebd-8e89-23c701e1b9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:17.785634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:17.786592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:17.788437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:17.789170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] PATH_INFO: `/v3/users/7127a5d0909a49b9b8ee4c472ac28012` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:17.792815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7127a5d0909a49b9b8ee4c472ac28012)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:17.797590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:17.850870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5f768d96-6bb2-4ebd-8e89-23c701e1b9bf None None] MFA Rules not processed for user `ae31b2903c8e458ca7c68082572db8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:17.855727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5f768d96-6bb2-4ebd-8e89-23c701e1b9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:18.034698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 840/1683] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:18.036365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f768d96-6bb2-4ebd-8e89-23c701e1b9bf None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.039197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5f768d96-6bb2-4ebd-8e89-23c701e1b9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:18.103774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:18.116921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7127a5d0909a49b9b8ee4c472ac28012'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.131896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f768d96-6bb2-4ebd-8e89-23c701e1b9bf None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:18.138571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:18.139095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.139377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.139651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] PATH_INFO: `/v3/users/793cffd0e0c04069ba94846af1139f2d` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.141420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793cffd0e0c04069ba94846af1139f2d)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:18.143935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:18.251035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7127a5d0909a49b9b8ee4c472ac28012'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.292632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7127a5d0909a49b9b8ee4c472ac28012'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.309405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7127a5d0909a49b9b8ee4c472ac28012'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.312004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7127a5d0909a49b9b8ee4c472ac28012'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.334189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 844/1684] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:17 2022] DELETE /identity/v3/users/7127a5d0909a49b9b8ee4c472ac28012 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:18.335544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.345962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:18.437647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793cffd0e0c04069ba94846af1139f2d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.506326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657181f-6b30-4cfc-941e-bc54bde5a08b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:18.519554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793cffd0e0c04069ba94846af1139f2d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.524948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b5acd-e195-4ff8-949b-3774bd7e276c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:18.528205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b5acd-e195-4ff8-949b-3774bd7e276c admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.528975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b5acd-e195-4ff8-949b-3774bd7e276c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.529636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b5acd-e195-4ff8-949b-3774bd7e276c admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.533188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b5acd-e195-4ff8-949b-3774bd7e276c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:18.537197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b5acd-e195-4ff8-949b-3774bd7e276c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:18.567336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793cffd0e0c04069ba94846af1139f2d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.582454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 845/1685] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:18.582602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793cffd0e0c04069ba94846af1139f2d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.584333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9834f3-c832-4da6-b06d-31a30d717b43 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.584900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9834f3-c832-4da6-b06d-31a30d717b43 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.585317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9834f3-c832-4da6-b06d-31a30d717b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.603643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8a48b02e-33c1-422f-a3b4-25286bbf99a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793cffd0e0c04069ba94846af1139f2d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:18.619568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-7c9834f3-c832-4da6-b06d-31a30d717b43 None None] MFA Rules not processed for user `ae31b2903c8e458ca7c68082572db8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:18.630520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 841/1686] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:18 2022] DELETE /identity/v3/users/793cffd0e0c04069ba94846af1139f2d => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:18.634101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f34a-9209-49bc-975d-8d0875d5d2ad None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:18.634578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f34a-9209-49bc-975d-8d0875d5d2ad None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:18.635162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f34a-9209-49bc-975d-8d0875d5d2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:18.637324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7c9834f3-c832-4da6-b06d-31a30d717b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:18.746172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4179f34a-9209-49bc-975d-8d0875d5d2ad None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:18.747784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4179f34a-9209-49bc-975d-8d0875d5d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:18.839766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 846/1687] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:18.840796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9834f3-c832-4da6-b06d-31a30d717b43 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:18.844048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7c9834f3-c832-4da6-b06d-31a30d717b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:19.017754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 842/1688] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:19.018739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179f34a-9209-49bc-975d-8d0875d5d2ad None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.022115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4179f34a-9209-49bc-975d-8d0875d5d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:19.068216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9834f3-c832-4da6-b06d-31a30d717b43 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.084507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4648c59-f014-42b1-bfde-de08a144f71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.086168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4648c59-f014-42b1-bfde-de08a144f71d admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.086457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4648c59-f014-42b1-bfde-de08a144f71d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.086807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4648c59-f014-42b1-bfde-de08a144f71d admin admin] PATH_INFO: `/v3/projects/668a1f69935c4da0823d39e382f11ce7/users/2b121d3f4bae46ec9e2d50c88a56764d/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.121324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:19.128282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4648c59-f014-42b1-bfde-de08a144f71d admin admin] RBAC: Authorizing `identity:create_grant(project_id=668a1f69935c4da0823d39e382f11ce7, user_id=2b121d3f4bae46ec9e2d50c88a56764d, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.136142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4648c59-f014-42b1-bfde-de08a144f71d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.232556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 847/1689] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:18 2022] PUT /identity/v3/projects/668a1f69935c4da0823d39e382f11ce7/users/2b121d3f4bae46ec9e2d50c88a56764d/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:19.233517 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4648c59-f014-42b1-bfde-de08a144f71d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.237247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d4648c59-f014-42b1-bfde-de08a144f71d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:19.262162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179f34a-9209-49bc-975d-8d0875d5d2ad None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.401138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4648c59-f014-42b1-bfde-de08a144f71d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.410394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.412005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.412779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.414657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] PATH_INFO: `/v3/projects/16c7db134ed345429ef0bdb3261328de` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.416526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c7db134ed345429ef0bdb3261328de)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.419430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.420473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.420694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.420964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.421970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.423192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:19.515150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.516806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.548551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 843/1690] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:19.552158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.557163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:19.606573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96401447-fc32-42aa-90e9-ab98a4a0028b service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.633612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c7db134ed345429ef0bdb3261328de', 'request_id': 'req-59581fd2-8138-4bed-8d0c-11474c95ce10'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:19.644897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c7db134ed345429ef0bdb3261328de', 'request_id': 'req-59581fd2-8138-4bed-8d0c-11474c95ce10'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:19.766414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c7db134ed345429ef0bdb3261328de', 'request_id': 'req-59581fd2-8138-4bed-8d0c-11474c95ce10'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:19.790029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] The token cache is being invalidate because project 16c7db134ed345429ef0bdb3261328de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:19.790537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:19.791451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:19.797823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 848/1691] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:19 2022] DELETE /identity/v3/projects/16c7db134ed345429ef0bdb3261328de => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:19.798287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.798968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.799484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.799764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] PATH_INFO: `/v3/users/4bbd40052949468a897744a3010f337c` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.800081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.801626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:19.801757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bbd40052949468a897744a3010f337c)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.804227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.824696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59581fd2-8138-4bed-8d0c-11474c95ce10 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:19.841561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf0b0-06e4-452f-b351-a4e9bcce26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:19.843097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf0b0-06e4-452f-b351-a4e9bcce26c2 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:19.843364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf0b0-06e4-452f-b351-a4e9bcce26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:19.843632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf0b0-06e4-452f-b351-a4e9bcce26c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:19.844758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf0b0-06e4-452f-b351-a4e9bcce26c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:19.846522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf0b0-06e4-452f-b351-a4e9bcce26c2 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:19.861821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 849/1692] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:19.863346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf0b0-06e4-452f-b351-a4e9bcce26c2 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:19.866485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bacaf0b0-06e4-452f-b351-a4e9bcce26c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:19.921380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:20.009424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:20.028197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bbd40052949468a897744a3010f337c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:20.035141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf0b0-06e4-452f-b351-a4e9bcce26c2 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:20.047754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70508495-5eeb-4cc7-aed2-d52a43c02fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:20.048700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70508495-5eeb-4cc7-aed2-d52a43c02fdf service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.049009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70508495-5eeb-4cc7-aed2-d52a43c02fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.050912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70508495-5eeb-4cc7-aed2-d52a43c02fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.070973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-70508495-5eeb-4cc7-aed2-d52a43c02fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.117666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70508495-5eeb-4cc7-aed2-d52a43c02fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:20.119535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70508495-5eeb-4cc7-aed2-d52a43c02fdf service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:20.124320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bbd40052949468a897744a3010f337c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:20.135867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bbd40052949468a897744a3010f337c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:20.156892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 850/1693] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:20.158992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2530e-9682-466b-b61c-3d0c34b8fa75 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.159327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2530e-9682-466b-b61c-3d0c34b8fa75 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.159696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2530e-9682-466b-b61c-3d0c34b8fa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.183444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bbd40052949468a897744a3010f337c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:20.186622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-21dea465-cb48-4c61-97c7-7313b606acfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bbd40052949468a897744a3010f337c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:20.188699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-e3c2530e-9682-466b-b61c-3d0c34b8fa75 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:20.189954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e3c2530e-9682-466b-b61c-3d0c34b8fa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.202653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 851/1694] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:20.203461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2530e-9682-466b-b61c-3d0c34b8fa75 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:20.203957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 844/1695] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:19 2022] DELETE /identity/v3/users/4bbd40052949468a897744a3010f337c => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:20.205371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373e4b1-9c42-4a1c-b539-8649b30bf8f2 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.205726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373e4b1-9c42-4a1c-b539-8649b30bf8f2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.206309 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373e4b1-9c42-4a1c-b539-8649b30bf8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.213458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e3c2530e-9682-466b-b61c-3d0c34b8fa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.243114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2530e-9682-466b-b61c-3d0c34b8fa75 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:20.290699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e373e4b1-9c42-4a1c-b539-8649b30bf8f2 None None] MFA Rules not processed for user `d6e9bd6db85b4decbdac1908912d158c`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:20.292178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e373e4b1-9c42-4a1c-b539-8649b30bf8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.347607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26d747-33cd-4371-9479-2cbbe986eb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:20.349705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26d747-33cd-4371-9479-2cbbe986eb9e admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.350062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26d747-33cd-4371-9479-2cbbe986eb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.350967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26d747-33cd-4371-9479-2cbbe986eb9e admin admin] PATH_INFO: `/v3/projects/668a1f69935c4da0823d39e382f11ce7/users/2b121d3f4bae46ec9e2d50c88a56764d/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.352647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26d747-33cd-4371-9479-2cbbe986eb9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=668a1f69935c4da0823d39e382f11ce7, user_id=2b121d3f4bae46ec9e2d50c88a56764d, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:20.355986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26d747-33cd-4371-9479-2cbbe986eb9e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:20.372265 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 852/1696] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:20 2022] PUT /identity/v3/projects/668a1f69935c4da0823d39e382f11ce7/users/2b121d3f4bae46ec9e2d50c88a56764d/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:20.374820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9e6f4-fb8d-42d7-8060-91552a000fd6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.375174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9e6f4-fb8d-42d7-8060-91552a000fd6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.375499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9e6f4-fb8d-42d7-8060-91552a000fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.417431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-74d9e6f4-fb8d-42d7-8060-91552a000fd6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:20.419149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-74d9e6f4-fb8d-42d7-8060-91552a000fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.429817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 845/1697] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:20.431402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85ebbe-aa57-436e-8c27-d5db8a0749c6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.431941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85ebbe-aa57-436e-8c27-d5db8a0749c6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.432289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85ebbe-aa57-436e-8c27-d5db8a0749c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.462690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8b85ebbe-aa57-436e-8c27-d5db8a0749c6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:20.471874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8b85ebbe-aa57-436e-8c27-d5db8a0749c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.560743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 853/1698] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:20.563289 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa6ba3-c23d-4a56-93d8-a69218b85788 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.563573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa6ba3-c23d-4a56-93d8-a69218b85788 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.565898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa6ba3-c23d-4a56-93d8-a69218b85788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.572446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 846/1699] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:20.589198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930a37-63c0-4946-a69e-42a2021cc850 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.589951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930a37-63c0-4946-a69e-42a2021cc850 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.590670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930a37-63c0-4946-a69e-42a2021cc850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.619117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2aaa6ba3-c23d-4a56-93d8-a69218b85788 None None] MFA Rules not processed for user `60dc4976d686449f80d3bfad387a2406`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:20.621329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2aaa6ba3-c23d-4a56-93d8-a69218b85788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.656885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b5930a37-63c0-4946-a69e-42a2021cc850 None None] MFA Rules not processed for user `2b121d3f4bae46ec9e2d50c88a56764d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:20.659441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b5930a37-63c0-4946-a69e-42a2021cc850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.708465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 854/1700] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:20.709975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec874c-ea6a-4b87-8f39-dccf4387c8dc None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.710328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec874c-ea6a-4b87-8f39-dccf4387c8dc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.710668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec874c-ea6a-4b87-8f39-dccf4387c8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.745434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-acec874c-ea6a-4b87-8f39-dccf4387c8dc None None] MFA Rules not processed for user `d6e9bd6db85b4decbdac1908912d158c`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:20.746748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-acec874c-ea6a-4b87-8f39-dccf4387c8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.793407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 847/1701] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:20.794328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5930a37-63c0-4946-a69e-42a2021cc850 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:20.821594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5930a37-63c0-4946-a69e-42a2021cc850 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:20.834262 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:20.835220 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.836623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.836814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.838959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:20.839765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:20.843042 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:20.857842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 855/1702] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:20.859582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec874c-ea6a-4b87-8f39-dccf4387c8dc None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:20.862361 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-acec874c-ea6a-4b87-8f39-dccf4387c8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.888194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 848/1703] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:20.888194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:20.889473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:20.962392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec874c-ea6a-4b87-8f39-dccf4387c8dc None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:20.968206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3833-6ca9-4ff6-a7ec-5355e3f8c57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:20.969056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3833-6ca9-4ff6-a7ec-5355e3f8c57b service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:20.969335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3833-6ca9-4ff6-a7ec-5355e3f8c57b service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:20.969646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3833-6ca9-4ff6-a7ec-5355e3f8c57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:20.984638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc3833-6ca9-4ff6-a7ec-5355e3f8c57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:20.986406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc3833-6ca9-4ff6-a7ec-5355e3f8c57b service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:21.013663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 856/1704] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:21.013663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3833-6ca9-4ff6-a7ec-5355e3f8c57b service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:21.014395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-76bc3833-6ca9-4ff6-a7ec-5355e3f8c57b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:21.036507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc3833-6ca9-4ff6-a7ec-5355e3f8c57b service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:21.045416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:21.047960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:21.047960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:21.047960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] PATH_INFO: `/v3/projects/95d311c9e53e4b2f89a1b7a95dd44dce` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:21.047960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d311c9e53e4b2f89a1b7a95dd44dce)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:21.050067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:21.118962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ea0a-e6a0-4a60-8471-a885f7a40743 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:21.125735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4fd0c-5f7d-4c12-a568-576caba330e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:21.126298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4fd0c-5f7d-4c12-a568-576caba330e8 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:21.126559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4fd0c-5f7d-4c12-a568-576caba330e8 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:21.126873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4fd0c-5f7d-4c12-a568-576caba330e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:21.157116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4fd0c-5f7d-4c12-a568-576caba330e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:21.160279 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4fd0c-5f7d-4c12-a568-576caba330e8 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:21.202073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 849/1705] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:21.210964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4fd0c-5f7d-4c12-a568-576caba330e8 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:21.212738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e9e4fd0c-5f7d-4c12-a568-576caba330e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:21.241730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d311c9e53e4b2f89a1b7a95dd44dce', 'request_id': 'req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:21.268410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d311c9e53e4b2f89a1b7a95dd44dce', 'request_id': 'req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:21.364965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d311c9e53e4b2f89a1b7a95dd44dce', 'request_id': 'req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:21.393767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] The token cache is being invalidate because project 95d311c9e53e4b2f89a1b7a95dd44dce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:21.394153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:21.396373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b572d2ae-eb69-4aeb-b9bd-bed8366ccb0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:21.398182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4fd0c-5f7d-4c12-a568-576caba330e8 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:21.403051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 857/1706] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:21 2022] DELETE /identity/v3/projects/95d311c9e53e4b2f89a1b7a95dd44dce => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:21.405918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6b421-7dd5-447c-a3d7-35544b7aa1f5 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:21.419426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6b421-7dd5-447c-a3d7-35544b7aa1f5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:21.419701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6b421-7dd5-447c-a3d7-35544b7aa1f5 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:21.423482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 858/1707] 10.222.0.57 () {54 vars in 912 bytes} [Mon Aug 8 17:00:21 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:00:21.424439 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6b421-7dd5-447c-a3d7-35544b7aa1f5 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:21.430072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7ab6b421-7dd5-447c-a3d7-35544b7aa1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:21.483788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:21.517113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:21.542334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:21.543310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:21.543763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:21.544040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:21.554287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:21.666059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6b421-7dd5-447c-a3d7-35544b7aa1f5 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:21.677924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d89ceb-e366-49f3-862d-71cb930c83f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:21.678885 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d89ceb-e366-49f3-862d-71cb930c83f0 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:21.679130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d89ceb-e366-49f3-862d-71cb930c83f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:21.679364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d89ceb-e366-49f3-862d-71cb930c83f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:21.679957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d89ceb-e366-49f3-862d-71cb930c83f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:21.681917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d89ceb-e366-49f3-862d-71cb930c83f0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:21.737523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:21.739803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:21.765938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 850/1708] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:21.770895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:21.772236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:21.790027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf4a7b-d6c8-4b6b-a2b6-d5d6c2fc9688 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:21.795999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:21.797530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:21.797949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:21.798306 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] PATH_INFO: `/v3/projects/a326b1ccd4d3402d9abdd4dcbf6e3f2c` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:21.800078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a326b1ccd4d3402d9abdd4dcbf6e3f2c)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:21.803082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:21.825845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 859/1709] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:21 2022] POST /identity/v3/projects => generated 365 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:21.825845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927726d9-3cad-4f9a-b9cc-7873a1e41d7f None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:21.825845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927726d9-3cad-4f9a-b9cc-7873a1e41d7f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:21.825845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927726d9-3cad-4f9a-b9cc-7873a1e41d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:21.859342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-927726d9-3cad-4f9a-b9cc-7873a1e41d7f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:21.860743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-927726d9-3cad-4f9a-b9cc-7873a1e41d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:21.993995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 860/1710] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:21.998102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cee01-71a9-4fee-a7c7-20ba224e2169 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:21.999338 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cee01-71a9-4fee-a7c7-20ba224e2169 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:22.000301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cee01-71a9-4fee-a7c7-20ba224e2169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:22.008034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a326b1ccd4d3402d9abdd4dcbf6e3f2c', 'request_id': 'req-2826eff1-1cef-4d20-9c9b-90b312560923'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:22.024052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a326b1ccd4d3402d9abdd4dcbf6e3f2c', 'request_id': 'req-2826eff1-1cef-4d20-9c9b-90b312560923'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:22.071211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1d2cee01-71a9-4fee-a7c7-20ba224e2169 None None] MFA Rules not processed for user `f50ee57e2418417cbe68915ca3f00f76`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:22.073617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1d2cee01-71a9-4fee-a7c7-20ba224e2169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:22.121662 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a326b1ccd4d3402d9abdd4dcbf6e3f2c', 'request_id': 'req-2826eff1-1cef-4d20-9c9b-90b312560923'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:22.147685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] The token cache is being invalidate because project a326b1ccd4d3402d9abdd4dcbf6e3f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:22.147931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:22.149365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:22.152814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 851/1711] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:21 2022] DELETE /identity/v3/projects/a326b1ccd4d3402d9abdd4dcbf6e3f2c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:22.154879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:22.158654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:22.174788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 861/1712] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:00:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:22.175684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cee01-71a9-4fee-a7c7-20ba224e2169 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:22.179944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1d2cee01-71a9-4fee-a7c7-20ba224e2169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:22.317058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826eff1-1cef-4d20-9c9b-90b312560923 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:22.324012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7aca0f-7d6d-45db-8be9-256bd175f0a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:22.324791 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aca0f-7d6d-45db-8be9-256bd175f0a0 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:22.325254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aca0f-7d6d-45db-8be9-256bd175f0a0 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:22.325595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aca0f-7d6d-45db-8be9-256bd175f0a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:22.328330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4d7aca0f-7d6d-45db-8be9-256bd175f0a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:22.351236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cee01-71a9-4fee-a7c7-20ba224e2169 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:22.358737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fc7dd-2cf2-498a-896e-767018c7ae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:22.359463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fc7dd-2cf2-498a-896e-767018c7ae45 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:22.359746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fc7dd-2cf2-498a-896e-767018c7ae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:22.360020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fc7dd-2cf2-498a-896e-767018c7ae45 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:22.361477 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fc7dd-2cf2-498a-896e-767018c7ae45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:22.371869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fc7dd-2cf2-498a-896e-767018c7ae45 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:22.481880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 862/1713] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:22 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:22.486290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fc7dd-2cf2-498a-896e-767018c7ae45 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:22.490441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ca0fc7dd-2cf2-498a-896e-767018c7ae45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:22.492605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aca0f-7d6d-45db-8be9-256bd175f0a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:22.493626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aca0f-7d6d-45db-8be9-256bd175f0a0 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:22.520379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 852/1714] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:22.522244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558b95f-7824-435f-aa3d-4bf3265399b5 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:22.522765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558b95f-7824-435f-aa3d-4bf3265399b5 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:22.523200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558b95f-7824-435f-aa3d-4bf3265399b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:22.542524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fc7dd-2cf2-498a-896e-767018c7ae45 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:22.556356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c558b95f-7824-435f-aa3d-4bf3265399b5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:22.579053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c558b95f-7824-435f-aa3d-4bf3265399b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:22.579559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:22.579559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:22.579559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:22.579559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] PATH_INFO: `/v3/projects/9a1e963209924abb967428435401540a` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:22.580069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1e963209924abb967428435401540a)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:22.580153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 853/1715] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:22.580240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:22.580323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558b95f-7824-435f-aa3d-4bf3265399b5 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:22.580323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c558b95f-7824-435f-aa3d-4bf3265399b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:22.730363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558b95f-7824-435f-aa3d-4bf3265399b5 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:22.735645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b82c4e-42e7-432a-96b6-e9e977ab414e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:22.736282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b82c4e-42e7-432a-96b6-e9e977ab414e service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:22.737296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b82c4e-42e7-432a-96b6-e9e977ab414e service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:22.737591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b82c4e-42e7-432a-96b6-e9e977ab414e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:22.739571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b4b82c4e-42e7-432a-96b6-e9e977ab414e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:22.852049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1e963209924abb967428435401540a', 'request_id': 'req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:22.885054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1e963209924abb967428435401540a', 'request_id': 'req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:22.896338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b82c4e-42e7-432a-96b6-e9e977ab414e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:22.906634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b82c4e-42e7-432a-96b6-e9e977ab414e service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:22.922589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 854/1716] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:22.932196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882e021-b82e-4f15-b8bb-c52e703bab8a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:22.932588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882e021-b82e-4f15-b8bb-c52e703bab8a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:22.932961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882e021-b82e-4f15-b8bb-c52e703bab8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:22.966587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6882e021-b82e-4f15-b8bb-c52e703bab8a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:22.967984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6882e021-b82e-4f15-b8bb-c52e703bab8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:22.980919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1e963209924abb967428435401540a', 'request_id': 'req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:23.029739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] The token cache is being invalidate because project 9a1e963209924abb967428435401540a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:23.038060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:23.038060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:23.052140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 863/1717] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:22 2022] DELETE /identity/v3/projects/9a1e963209924abb967428435401540a => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:23.053184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:23.063992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:23.144965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 855/1718] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:23.146140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882e021-b82e-4f15-b8bb-c52e703bab8a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:23.152923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6882e021-b82e-4f15-b8bb-c52e703bab8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:23.168868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdca2f2-8d38-422c-ad6c-4a0ba31e34f4 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:23.176560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:23.179168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.181532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882e021-b82e-4f15-b8bb-c52e703bab8a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:23.181665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.181665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.184295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:23.191751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a2a74-4252-4935-a207-ec14da75be53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:23.192497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a2a74-4252-4935-a207-ec14da75be53 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.194680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a2a74-4252-4935-a207-ec14da75be53 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.195459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a2a74-4252-4935-a207-ec14da75be53 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.199011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a2a74-4252-4935-a207-ec14da75be53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:23.199175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:23.200896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a2a74-4252-4935-a207-ec14da75be53 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:23.201019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:23.213180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 864/1719] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:23.214174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:23.215953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:23.226978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 856/1720] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:23.228764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50cb47-73c5-42ae-a5d5-b775ec733899 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.229158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50cb47-73c5-42ae-a5d5-b775ec733899 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.229588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50cb47-73c5-42ae-a5d5-b775ec733899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.258662 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6c50cb47-73c5-42ae-a5d5-b775ec733899 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:23.260353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6c50cb47-73c5-42ae-a5d5-b775ec733899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:23.261049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564f35c-4111-446e-9559-9173cc354c6c service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:23.271312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 857/1721] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:23.271531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f2f3b-e920-4f61-ad58-e77b8a81f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:23.272120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f2f3b-e920-4f61-ad58-e77b8a81f4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.272578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f2f3b-e920-4f61-ad58-e77b8a81f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.272860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f2f3b-e920-4f61-ad58-e77b8a81f4f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.273575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f2f3b-e920-4f61-ad58-e77b8a81f4f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:23.273930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f2f3b-e920-4f61-ad58-e77b8a81f4f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:23.279014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce900a7-3960-44ed-8ab2-e3ec22ca64c7 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.279014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce900a7-3960-44ed-8ab2-e3ec22ca64c7 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.279014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce900a7-3960-44ed-8ab2-e3ec22ca64c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.279277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f2f3b-e920-4f61-ad58-e77b8a81f4f5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:23.316298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 865/1722] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:23.317408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f2f3b-e920-4f61-ad58-e77b8a81f4f5 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:23.332735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1ce900a7-3960-44ed-8ab2-e3ec22ca64c7 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:23.341639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1ce900a7-3960-44ed-8ab2-e3ec22ca64c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:23.344493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f2f3b-e920-4f61-ad58-e77b8a81f4f5 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:23.349938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22610877-086f-4dd4-9621-22678c4d5952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:23.351585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22610877-086f-4dd4-9621-22678c4d5952 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.358852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22610877-086f-4dd4-9621-22678c4d5952 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.359221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22610877-086f-4dd4-9621-22678c4d5952 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.365146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 858/1723] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:23.365146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602a631-5444-429f-bab1-a364533ef393 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.365146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602a631-5444-429f-bab1-a364533ef393 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.365146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602a631-5444-429f-bab1-a364533ef393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.381703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22610877-086f-4dd4-9621-22678c4d5952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:23.386763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22610877-086f-4dd4-9621-22678c4d5952 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:23.388183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d602a631-5444-429f-bab1-a364533ef393 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:23.389493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d602a631-5444-429f-bab1-a364533ef393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:23.401519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 859/1724] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:23.412127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602a631-5444-429f-bab1-a364533ef393 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:23.432741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602a631-5444-429f-bab1-a364533ef393 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:23.438388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497c242-ac5f-400a-a613-64d39fa3356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:23.439895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497c242-ac5f-400a-a613-64d39fa3356f admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.440280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497c242-ac5f-400a-a613-64d39fa3356f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.440727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497c242-ac5f-400a-a613-64d39fa3356f admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.441467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497c242-ac5f-400a-a613-64d39fa3356f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:23.449029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5497c242-ac5f-400a-a613-64d39fa3356f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:23.449505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 866/1725] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:23.451475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22610877-086f-4dd4-9621-22678c4d5952 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:23.472817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22610877-086f-4dd4-9621-22678c4d5952 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:23.481332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c529534-c91b-47c4-bcdf-697277bcf01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:23.483171 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c529534-c91b-47c4-bcdf-697277bcf01b admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.484435 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c529534-c91b-47c4-bcdf-697277bcf01b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.484875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c529534-c91b-47c4-bcdf-697277bcf01b admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.485721 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c529534-c91b-47c4-bcdf-697277bcf01b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:23.486235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c529534-c91b-47c4-bcdf-697277bcf01b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:23.489625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c529534-c91b-47c4-bcdf-697277bcf01b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:23.567253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 867/1726] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:23.569039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c529534-c91b-47c4-bcdf-697277bcf01b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:23.607822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 860/1727] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:23 2022] POST /identity/v3/projects => generated 367 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:23.608776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497c242-ac5f-400a-a613-64d39fa3356f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:23.611257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c529534-c91b-47c4-bcdf-697277bcf01b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:23.681317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:23.728238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497c242-ac5f-400a-a613-64d39fa3356f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:23.734495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea9e9e-c7cd-4f21-acf3-983ab5bcc12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:23.735335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea9e9e-c7cd-4f21-acf3-983ab5bcc12f admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.735622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea9e9e-c7cd-4f21-acf3-983ab5bcc12f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.736431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea9e9e-c7cd-4f21-acf3-983ab5bcc12f admin admin] PATH_INFO: `/v3/projects/226a05f0ef6546b7a90c6146608089e6/users/1c8b1ce87d074acd8d608575d3b5c7cd/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.764043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea9e9e-c7cd-4f21-acf3-983ab5bcc12f admin admin] RBAC: Authorizing `identity:create_grant(project_id=226a05f0ef6546b7a90c6146608089e6, user_id=1c8b1ce87d074acd8d608575d3b5c7cd, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:23.768485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea9e9e-c7cd-4f21-acf3-983ab5bcc12f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:23.781625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:23.814373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4dbaf-04b3-4798-88ac-da8850bfe58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:23.815008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4dbaf-04b3-4798-88ac-da8850bfe58d admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.815320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4dbaf-04b3-4798-88ac-da8850bfe58d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.815596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4dbaf-04b3-4798-88ac-da8850bfe58d admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.816379 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4dbaf-04b3-4798-88ac-da8850bfe58d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:23.816689 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4dbaf-04b3-4798-88ac-da8850bfe58d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:23.818381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4dbaf-04b3-4798-88ac-da8850bfe58d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:23.832759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 868/1728] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:23.834221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7defe-fa7b-4a26-a915-1fd942511216 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.835017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7defe-fa7b-4a26-a915-1fd942511216 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.835295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7defe-fa7b-4a26-a915-1fd942511216 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.837585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 869/1729] 10.222.0.57 () {54 vars in 912 bytes} [Mon Aug 8 17:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:00:23.838082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7defe-fa7b-4a26-a915-1fd942511216 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:23.872394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7defe-fa7b-4a26-a915-1fd942511216 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:23.880033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41a0e5-6d43-4d3e-8230-5af1612c166a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:23.881043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41a0e5-6d43-4d3e-8230-5af1612c166a admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:23.881424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41a0e5-6d43-4d3e-8230-5af1612c166a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:23.881749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41a0e5-6d43-4d3e-8230-5af1612c166a admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:23.885198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41a0e5-6d43-4d3e-8230-5af1612c166a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:23.885832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41a0e5-6d43-4d3e-8230-5af1612c166a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:23.921598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 861/1730] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:23 2022] PUT /identity/v3/projects/226a05f0ef6546b7a90c6146608089e6/users/1c8b1ce87d074acd8d608575d3b5c7cd/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:23.921598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea9e9e-c7cd-4f21-acf3-983ab5bcc12f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.027098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea9e9e-c7cd-4f21-acf3-983ab5bcc12f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.039032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716cf14-ebb8-456b-9ef3-231fb3e43020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.039843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716cf14-ebb8-456b-9ef3-231fb3e43020 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.040119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716cf14-ebb8-456b-9ef3-231fb3e43020 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.040670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716cf14-ebb8-456b-9ef3-231fb3e43020 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.041019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716cf14-ebb8-456b-9ef3-231fb3e43020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.043153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716cf14-ebb8-456b-9ef3-231fb3e43020 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.066758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 870/1731] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:23 2022] POST /identity/v3/projects => generated 365 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:24.067670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41a0e5-6d43-4d3e-8230-5af1612c166a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.152243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41a0e5-6d43-4d3e-8230-5af1612c166a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.157768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b9ca8-1b4b-42c4-bc81-2da911fea9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.158470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b9ca8-1b4b-42c4-bc81-2da911fea9fe admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.162288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b9ca8-1b4b-42c4-bc81-2da911fea9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.162288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b9ca8-1b4b-42c4-bc81-2da911fea9fe admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.162288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b9ca8-1b4b-42c4-bc81-2da911fea9fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.162288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b9ca8-1b4b-42c4-bc81-2da911fea9fe admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.162760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 862/1732] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:23 2022] POST /identity/v3/users => generated 284 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:24.165521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43017a9a-365a-490f-b5db-164a7ce92300 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.166143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43017a9a-365a-490f-b5db-164a7ce92300 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.166575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43017a9a-365a-490f-b5db-164a7ce92300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.207164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-43017a9a-365a-490f-b5db-164a7ce92300 None None] MFA Rules not processed for user `f50ee57e2418417cbe68915ca3f00f76`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:24.210809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-43017a9a-365a-490f-b5db-164a7ce92300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:24.283540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 863/1733] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:00:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:24.284853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140d226-0a61-40c2-bbe7-ab072b7c98e9 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.285204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140d226-0a61-40c2-bbe7-ab072b7c98e9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.285738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140d226-0a61-40c2-bbe7-ab072b7c98e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.311572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 871/1734] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:24 2022] POST /identity/v3/projects => generated 373 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:24.311572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b9ca8-1b4b-42c4-bc81-2da911fea9fe admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.345876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f140d226-0a61-40c2-bbe7-ab072b7c98e9 None None] MFA Rules not processed for user `1c8b1ce87d074acd8d608575d3b5c7cd`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:24.347125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f140d226-0a61-40c2-bbe7-ab072b7c98e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:24.448076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 864/1735] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:24.451961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140d226-0a61-40c2-bbe7-ab072b7c98e9 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.498636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b9ca8-1b4b-42c4-bc81-2da911fea9fe admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.504386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4e982-99b3-4c20-b13a-c0cd9b4e3cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.505325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4e982-99b3-4c20-b13a-c0cd9b4e3cef admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.505889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4e982-99b3-4c20-b13a-c0cd9b4e3cef admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.506412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4e982-99b3-4c20-b13a-c0cd9b4e3cef admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.507287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4e982-99b3-4c20-b13a-c0cd9b4e3cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.509565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4e982-99b3-4c20-b13a-c0cd9b4e3cef admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.556680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140d226-0a61-40c2-bbe7-ab072b7c98e9 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.563011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1c490-b10b-4df5-a5c7-68b8bbf82c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.564870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1c490-b10b-4df5-a5c7-68b8bbf82c4c admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.564870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1c490-b10b-4df5-a5c7-68b8bbf82c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.565187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1c490-b10b-4df5-a5c7-68b8bbf82c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.566316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1c490-b10b-4df5-a5c7-68b8bbf82c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.568345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1c490-b10b-4df5-a5c7-68b8bbf82c4c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.596711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 865/1736] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:24.597739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1c490-b10b-4df5-a5c7-68b8bbf82c4c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.601147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2ef1c490-b10b-4df5-a5c7-68b8bbf82c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:24.607824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 872/1737] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:24 2022] POST /identity/v3/users => generated 283 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:24.610208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4e982-99b3-4c20-b13a-c0cd9b4e3cef admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.642735 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4e982-99b3-4c20-b13a-c0cd9b4e3cef admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.647900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637676d-4014-40a7-b2db-026f093181dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.648505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637676d-4014-40a7-b2db-026f093181dd admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.648784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637676d-4014-40a7-b2db-026f093181dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.649052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637676d-4014-40a7-b2db-026f093181dd admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.649929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637676d-4014-40a7-b2db-026f093181dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.652016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637676d-4014-40a7-b2db-026f093181dd admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.721768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1c490-b10b-4df5-a5c7-68b8bbf82c4c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.732600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3f6ba-3290-4b0e-bdd6-0427228fe466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.733762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3f6ba-3290-4b0e-bdd6-0427228fe466 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.734226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3f6ba-3290-4b0e-bdd6-0427228fe466 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.734612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3f6ba-3290-4b0e-bdd6-0427228fe466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.747433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3f6ba-3290-4b0e-bdd6-0427228fe466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.749651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3f6ba-3290-4b0e-bdd6-0427228fe466 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.767501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 873/1739] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:24 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:24.767712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 866/1739] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:24.768733 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637676d-4014-40a7-b2db-026f093181dd admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.769557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c123bb2-6bc5-4bc0-b426-76cb5ca5a7f3 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.769889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c123bb2-6bc5-4bc0-b426-76cb5ca5a7f3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.770165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c123bb2-6bc5-4bc0-b426-76cb5ca5a7f3 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.776758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 867/1740] 10.222.0.57 () {54 vars in 961 bytes} [Mon Aug 8 17:00:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:00:24.779288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c123bb2-6bc5-4bc0-b426-76cb5ca5a7f3 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.790011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637676d-4014-40a7-b2db-026f093181dd admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.799819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c123bb2-6bc5-4bc0-b426-76cb5ca5a7f3 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.804296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd1b02-d758-46f2-aaf3-b798da0096b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.813368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd1b02-d758-46f2-aaf3-b798da0096b3 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.813696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651706ee-7c2d-4b6b-8af2-13eaaa575143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.813696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651706ee-7c2d-4b6b-8af2-13eaaa575143 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.813911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd1b02-d758-46f2-aaf3-b798da0096b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.813993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651706ee-7c2d-4b6b-8af2-13eaaa575143 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.814070 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd1b02-d758-46f2-aaf3-b798da0096b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.814147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651706ee-7c2d-4b6b-8af2-13eaaa575143 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.814230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd1b02-d758-46f2-aaf3-b798da0096b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.814323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651706ee-7c2d-4b6b-8af2-13eaaa575143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.814323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651706ee-7c2d-4b6b-8af2-13eaaa575143 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.815818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd1b02-d758-46f2-aaf3-b798da0096b3 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.825753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 868/1741] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:24.826635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651706ee-7c2d-4b6b-8af2-13eaaa575143 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.854593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651706ee-7c2d-4b6b-8af2-13eaaa575143 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.862852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf74f45-742f-42dc-8a86-69bc3b30b639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.864810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf74f45-742f-42dc-8a86-69bc3b30b639 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.864810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf74f45-742f-42dc-8a86-69bc3b30b639 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.864810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf74f45-742f-42dc-8a86-69bc3b30b639 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.866599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf74f45-742f-42dc-8a86-69bc3b30b639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.870791 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf74f45-742f-42dc-8a86-69bc3b30b639 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.894223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 869/1742] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:24.895332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf74f45-742f-42dc-8a86-69bc3b30b639 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.919960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf74f45-742f-42dc-8a86-69bc3b30b639 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:24.927138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51240d75-fbd8-475d-8929-fb3795e7ab1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:24.927907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51240d75-fbd8-475d-8929-fb3795e7ab1d admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.928257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51240d75-fbd8-475d-8929-fb3795e7ab1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.929024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51240d75-fbd8-475d-8929-fb3795e7ab1d admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.929775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51240d75-fbd8-475d-8929-fb3795e7ab1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:24.931440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51240d75-fbd8-475d-8929-fb3795e7ab1d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:24.947957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 874/1743] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:24 2022] POST /identity/v3/projects => generated 365 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:24.948602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 870/1744] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:24.949618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd1b02-d758-46f2-aaf3-b798da0096b3 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:24.949969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c509161-ba6f-4715-929a-74783b802408 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:24.950257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c509161-ba6f-4715-929a-74783b802408 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:24.950610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c509161-ba6f-4715-929a-74783b802408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:24.982577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8c509161-ba6f-4715-929a-74783b802408 None None] MFA Rules not processed for user `5853f9b60cf34380b218a5cda94f587d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:24.983838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8c509161-ba6f-4715-929a-74783b802408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:25.101829 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd1b02-d758-46f2-aaf3-b798da0096b3 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:25.111452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2085ce-ee38-4f19-9ee5-9a1dbc1f82a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:25.112131 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2085ce-ee38-4f19-9ee5-9a1dbc1f82a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.112833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2085ce-ee38-4f19-9ee5-9a1dbc1f82a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.113266 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2085ce-ee38-4f19-9ee5-9a1dbc1f82a6 admin admin] PATH_INFO: `/v3/projects/106c6c134f7b418699e3097adb022604/users/6f2f1e72bab542648a9142d974ec5d03/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.131658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 871/1745] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Aug 8 17:00:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:25.132477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c509161-ba6f-4715-929a-74783b802408 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:25.143237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2085ce-ee38-4f19-9ee5-9a1dbc1f82a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=106c6c134f7b418699e3097adb022604, user_id=6f2f1e72bab542648a9142d974ec5d03, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:25.144610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c509161-ba6f-4715-929a-74783b802408 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:25.146629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2085ce-ee38-4f19-9ee5-9a1dbc1f82a6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:25.159192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe653e-6412-4150-8a7a-255cdbc12294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:25.159926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe653e-6412-4150-8a7a-255cdbc12294 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.160104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe653e-6412-4150-8a7a-255cdbc12294 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.161378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe653e-6412-4150-8a7a-255cdbc12294 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.163834 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe653e-6412-4150-8a7a-255cdbc12294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:25.166442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe653e-6412-4150-8a7a-255cdbc12294 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:25.183787 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 872/1746] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:25.184822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe653e-6412-4150-8a7a-255cdbc12294 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:25.196557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe653e-6412-4150-8a7a-255cdbc12294 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:25.202054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b0a58-7f6f-4f33-a277-248322584ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:25.202696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b0a58-7f6f-4f33-a277-248322584ecf admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.202970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b0a58-7f6f-4f33-a277-248322584ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.203238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b0a58-7f6f-4f33-a277-248322584ecf admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.203861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b0a58-7f6f-4f33-a277-248322584ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:25.205875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b0a58-7f6f-4f33-a277-248322584ecf admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:25.279635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 875/1747] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:24 2022] PUT /identity/v3/projects/106c6c134f7b418699e3097adb022604/users/6f2f1e72bab542648a9142d974ec5d03/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:25.279635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2085ce-ee38-4f19-9ee5-9a1dbc1f82a6 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:25.362487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 873/1748] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:25 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:25.362487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b0a58-7f6f-4f33-a277-248322584ecf admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:25.381415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b0a58-7f6f-4f33-a277-248322584ecf admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:25.388062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2085ce-ee38-4f19-9ee5-9a1dbc1f82a6 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:25.410153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313495-1e95-4ac0-87bb-85aee0aff3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:25.411193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313495-1e95-4ac0-87bb-85aee0aff3fc admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.411742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313495-1e95-4ac0-87bb-85aee0aff3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.412297 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313495-1e95-4ac0-87bb-85aee0aff3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.414484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47313495-1e95-4ac0-87bb-85aee0aff3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:25.418655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47313495-1e95-4ac0-87bb-85aee0aff3fc admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:25.453266 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 876/1749] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:25.454240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313495-1e95-4ac0-87bb-85aee0aff3fc admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:25.471074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313495-1e95-4ac0-87bb-85aee0aff3fc admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:25.488854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721996ac-9142-4865-880a-3c1f9f115f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:25.489698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721996ac-9142-4865-880a-3c1f9f115f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.496052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721996ac-9142-4865-880a-3c1f9f115f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.496052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721996ac-9142-4865-880a-3c1f9f115f1b admin admin] PATH_INFO: `/v3/projects/2376b9d6148d496bb88cdba4cf2a77a9/users/25b6a59596144263a9bd0b798886b0c9/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.510189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e475d6d-c848-4477-8a30-bca53fda716f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:25.510915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721996ac-9142-4865-880a-3c1f9f115f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2376b9d6148d496bb88cdba4cf2a77a9, user_id=25b6a59596144263a9bd0b798886b0c9, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:25.511231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e475d6d-c848-4477-8a30-bca53fda716f service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.511520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e475d6d-c848-4477-8a30-bca53fda716f service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.511801 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e475d6d-c848-4477-8a30-bca53fda716f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.517745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721996ac-9142-4865-880a-3c1f9f115f1b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:25.518775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6e475d6d-c848-4477-8a30-bca53fda716f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:25.535908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e475d6d-c848-4477-8a30-bca53fda716f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:25.538853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e475d6d-c848-4477-8a30-bca53fda716f service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:25.574777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 874/1750] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:25.578055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3b691-5b65-430f-af37-33768e4b1418 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.578774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3b691-5b65-430f-af37-33768e4b1418 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.579053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3b691-5b65-430f-af37-33768e4b1418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.633478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-0df3b691-5b65-430f-af37-33768e4b1418 None None] MFA Rules not processed for user `6f2f1e72bab542648a9142d974ec5d03`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:25.637904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0df3b691-5b65-430f-af37-33768e4b1418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:25.642596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 877/1751] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:25 2022] PUT /identity/v3/projects/2376b9d6148d496bb88cdba4cf2a77a9/users/25b6a59596144263a9bd0b798886b0c9/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:25.643564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721996ac-9142-4865-880a-3c1f9f115f1b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:25.805541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 875/1752] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:25.806840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3b691-5b65-430f-af37-33768e4b1418 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:25.824652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3b691-5b65-430f-af37-33768e4b1418 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:25.832128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721996ac-9142-4865-880a-3c1f9f115f1b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:25.843938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792d389-000d-48f5-ad82-d78e234e9036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:25.844748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792d389-000d-48f5-ad82-d78e234e9036 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.845142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792d389-000d-48f5-ad82-d78e234e9036 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.845517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792d389-000d-48f5-ad82-d78e234e9036 admin admin] PATH_INFO: `/v3/projects/9ed2adbf8e814dc2894fe0e8141edd4c/users/c7ff504bc358424c9271f629641b7209/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.871509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792d389-000d-48f5-ad82-d78e234e9036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed2adbf8e814dc2894fe0e8141edd4c, user_id=c7ff504bc358424c9271f629641b7209, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:25.874775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792d389-000d-48f5-ad82-d78e234e9036 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:25.887917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2beb14f-95fa-4a26-8554-0930a733636c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:25.888593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2beb14f-95fa-4a26-8554-0930a733636c admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.888875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2beb14f-95fa-4a26-8554-0930a733636c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.889150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2beb14f-95fa-4a26-8554-0930a733636c admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.889932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2beb14f-95fa-4a26-8554-0930a733636c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:25.891476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2beb14f-95fa-4a26-8554-0930a733636c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:25.924607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 878/1753] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:25.926120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5c78c-dd47-439e-b707-16e5b353e9ab None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:25.930876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5c78c-dd47-439e-b707-16e5b353e9ab None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:25.931164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5c78c-dd47-439e-b707-16e5b353e9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:25.968704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 876/1754] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:25 2022] PUT /identity/v3/projects/9ed2adbf8e814dc2894fe0e8141edd4c/users/c7ff504bc358424c9271f629641b7209/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:25.969988 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792d389-000d-48f5-ad82-d78e234e9036 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:25.974190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7792d389-000d-48f5-ad82-d78e234e9036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:25.981270 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-74d5c78c-dd47-439e-b707-16e5b353e9ab None None] MFA Rules not processed for user `d6e9bd6db85b4decbdac1908912d158c`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:25.983144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-74d5c78c-dd47-439e-b707-16e5b353e9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:26.087815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 879/1755] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:26.087815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a6021-1fde-4155-af91-651510e79f8b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:26.087815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a6021-1fde-4155-af91-651510e79f8b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:26.087815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a6021-1fde-4155-af91-651510e79f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:26.121841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-054a6021-1fde-4155-af91-651510e79f8b None None] MFA Rules not processed for user `25b6a59596144263a9bd0b798886b0c9`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:26.123064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-054a6021-1fde-4155-af91-651510e79f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:26.182522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792d389-000d-48f5-ad82-d78e234e9036 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:26.211221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070dacf-c993-47e7-830e-3decfac1d79a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:26.211982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070dacf-c993-47e7-830e-3decfac1d79a service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:26.212367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070dacf-c993-47e7-830e-3decfac1d79a service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:26.212873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070dacf-c993-47e7-830e-3decfac1d79a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:26.229640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070dacf-c993-47e7-830e-3decfac1d79a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:26.230800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070dacf-c993-47e7-830e-3decfac1d79a service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:26.245873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 877/1756] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:26.251924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed65ff7-3df3-43cd-8862-71148e1353a9 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:26.252362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed65ff7-3df3-43cd-8862-71148e1353a9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:26.252773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed65ff7-3df3-43cd-8862-71148e1353a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:26.263214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 880/1757] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:26.267759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a6021-1fde-4155-af91-651510e79f8b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:26.287270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-7ed65ff7-3df3-43cd-8862-71148e1353a9 None None] MFA Rules not processed for user `60dc4976d686449f80d3bfad387a2406`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:26.288573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7ed65ff7-3df3-43cd-8862-71148e1353a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:26.365269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a6021-1fde-4155-af91-651510e79f8b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:26.376926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ef9c1-079b-4ea2-9bd3-a834518555a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:26.377084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ef9c1-079b-4ea2-9bd3-a834518555a8 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:26.377507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ef9c1-079b-4ea2-9bd3-a834518555a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:26.377908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ef9c1-079b-4ea2-9bd3-a834518555a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:26.378707 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ef9c1-079b-4ea2-9bd3-a834518555a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:26.380998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ef9c1-079b-4ea2-9bd3-a834518555a8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:26.434541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 878/1758] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:26.435589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed65ff7-3df3-43cd-8862-71148e1353a9 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:26.473927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed65ff7-3df3-43cd-8862-71148e1353a9 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:26.491758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fca32-0a06-4c29-bd72-4b4432d961af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:26.493213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fca32-0a06-4c29-bd72-4b4432d961af admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:26.493615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fca32-0a06-4c29-bd72-4b4432d961af admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:26.494159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fca32-0a06-4c29-bd72-4b4432d961af admin admin] PATH_INFO: `/v3/projects/080efe4258ee404cb87fb83ec6a6aa95/users/569f41455cf948cfb39c90e6ffcc8fca/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:26.512904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fca32-0a06-4c29-bd72-4b4432d961af admin admin] RBAC: Authorizing `identity:create_grant(project_id=080efe4258ee404cb87fb83ec6a6aa95, user_id=569f41455cf948cfb39c90e6ffcc8fca, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:26.515608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fca32-0a06-4c29-bd72-4b4432d961af admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:26.578531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 881/1759] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:26 2022] POST /identity/v3/projects => generated 363 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:26.602549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc910-a228-4646-bff2-1a398718a9aa None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:26.617302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc910-a228-4646-bff2-1a398718a9aa None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:26.617302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc910-a228-4646-bff2-1a398718a9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:26.656360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 879/1760] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:26 2022] PUT /identity/v3/projects/080efe4258ee404cb87fb83ec6a6aa95/users/569f41455cf948cfb39c90e6ffcc8fca/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:26.658387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fca32-0a06-4c29-bd72-4b4432d961af admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:26.661156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d4bfc910-a228-4646-bff2-1a398718a9aa None None] MFA Rules not processed for user `c7ff504bc358424c9271f629641b7209`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:26.664117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d4bfc910-a228-4646-bff2-1a398718a9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:26.810830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 882/1761] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:26.813168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfc910-a228-4646-bff2-1a398718a9aa None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:26.848905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fca32-0a06-4c29-bd72-4b4432d961af admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:26.864669 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03f77f-20ae-456d-9fa0-69b91b102abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:26.869685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03f77f-20ae-456d-9fa0-69b91b102abf admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:26.869685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03f77f-20ae-456d-9fa0-69b91b102abf admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:26.869685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03f77f-20ae-456d-9fa0-69b91b102abf admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:26.869685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03f77f-20ae-456d-9fa0-69b91b102abf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:26.871352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03f77f-20ae-456d-9fa0-69b91b102abf admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:26.963221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfc910-a228-4646-bff2-1a398718a9aa None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:26.971880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b2105-537c-435c-9127-eea6c3470c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:26.972578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b2105-537c-435c-9127-eea6c3470c74 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:26.972918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b2105-537c-435c-9127-eea6c3470c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:26.973257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b2105-537c-435c-9127-eea6c3470c74 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:26.974239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b2105-537c-435c-9127-eea6c3470c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:26.981570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b2105-537c-435c-9127-eea6c3470c74 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:27.004492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:27.142459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:27.149998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 880/1762] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:26 2022] POST /identity/v3/projects => generated 367 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:27.149998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03f77f-20ae-456d-9fa0-69b91b102abf admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:27.150339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 883/1763] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:26 2022] POST /identity/v3/users => generated 288 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:27.154003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b2105-537c-435c-9127-eea6c3470c74 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:27.169015 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b2105-537c-435c-9127-eea6c3470c74 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:27.263612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bd0ac-1869-4523-8f76-6389476688d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:27.264793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bd0ac-1869-4523-8f76-6389476688d4 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.265356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bd0ac-1869-4523-8f76-6389476688d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.265971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bd0ac-1869-4523-8f76-6389476688d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:27.267418 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bd0ac-1869-4523-8f76-6389476688d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:27.270621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bd0ac-1869-4523-8f76-6389476688d4 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:27.297462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 884/1764] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:00:27.299321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bd0ac-1869-4523-8f76-6389476688d4 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:27.319420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bd0ac-1869-4523-8f76-6389476688d4 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:27.323210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03f77f-20ae-456d-9fa0-69b91b102abf admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:27.329329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18288ddc-58c9-4f7e-aa3b-adabdea78613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:27.330582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18288ddc-58c9-4f7e-aa3b-adabdea78613 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.331263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18288ddc-58c9-4f7e-aa3b-adabdea78613 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.331831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18288ddc-58c9-4f7e-aa3b-adabdea78613 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:27.332326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001e81-ec4a-44e8-9bc7-ba98f567a281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:27.333308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18288ddc-58c9-4f7e-aa3b-adabdea78613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:27.334017 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001e81-ec4a-44e8-9bc7-ba98f567a281 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.335441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001e81-ec4a-44e8-9bc7-ba98f567a281 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.336107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001e81-ec4a-44e8-9bc7-ba98f567a281 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:27.337747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08001e81-ec4a-44e8-9bc7-ba98f567a281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:27.338978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18288ddc-58c9-4f7e-aa3b-adabdea78613 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:27.341091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08001e81-ec4a-44e8-9bc7-ba98f567a281 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:27.361423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 881/1765] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:27.366997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001e81-ec4a-44e8-9bc7-ba98f567a281 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:27.382803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001e81-ec4a-44e8-9bc7-ba98f567a281 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:27.395806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63619f26-cd06-44fc-b24f-e0a141cfd820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:27.396533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63619f26-cd06-44fc-b24f-e0a141cfd820 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.397511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63619f26-cd06-44fc-b24f-e0a141cfd820 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.397846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63619f26-cd06-44fc-b24f-e0a141cfd820 admin admin] PATH_INFO: `/v3/projects/0c652b3ae0534b9489f6374feff11a28/users/a9de993a28d54f5f87d72c90ad4d908a/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:27.420646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63619f26-cd06-44fc-b24f-e0a141cfd820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c652b3ae0534b9489f6374feff11a28, user_id=a9de993a28d54f5f87d72c90ad4d908a, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:27.423558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63619f26-cd06-44fc-b24f-e0a141cfd820 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:27.477921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 885/1766] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:27 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:27.478864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18288ddc-58c9-4f7e-aa3b-adabdea78613 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:27.480492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-18288ddc-58c9-4f7e-aa3b-adabdea78613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:27.522581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 882/1767] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:27 2022] PUT /identity/v3/projects/0c652b3ae0534b9489f6374feff11a28/users/a9de993a28d54f5f87d72c90ad4d908a/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:27.531138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63619f26-cd06-44fc-b24f-e0a141cfd820 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:27.673676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18288ddc-58c9-4f7e-aa3b-adabdea78613 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:27.722402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63619f26-cd06-44fc-b24f-e0a141cfd820 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:27.728327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942faeca-cb2a-467f-896e-45f7d55c6af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:27.729239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942faeca-cb2a-467f-896e-45f7d55c6af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.729671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942faeca-cb2a-467f-896e-45f7d55c6af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.730255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942faeca-cb2a-467f-896e-45f7d55c6af0 admin admin] PATH_INFO: `/v3/projects/080efe4258ee404cb87fb83ec6a6aa95/users/569f41455cf948cfb39c90e6ffcc8fca/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:27.732563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942faeca-cb2a-467f-896e-45f7d55c6af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=080efe4258ee404cb87fb83ec6a6aa95, user_id=569f41455cf948cfb39c90e6ffcc8fca, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:27.736548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942faeca-cb2a-467f-896e-45f7d55c6af0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:27.752013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 883/1768] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:27 2022] PUT /identity/v3/projects/080efe4258ee404cb87fb83ec6a6aa95/users/569f41455cf948cfb39c90e6ffcc8fca/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:27.753131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942faeca-cb2a-467f-896e-45f7d55c6af0 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:27.798880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0462bb3-0d5e-4dfa-ad4e-8fdb3326d693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:27.800213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0462bb3-0d5e-4dfa-ad4e-8fdb3326d693 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.800213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0462bb3-0d5e-4dfa-ad4e-8fdb3326d693 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.800401 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0462bb3-0d5e-4dfa-ad4e-8fdb3326d693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:27.875014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942faeca-cb2a-467f-896e-45f7d55c6af0 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:27.881023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b2171-e97a-4b6b-8f1f-52f6a9e51b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:27.882364 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b2171-e97a-4b6b-8f1f-52f6a9e51b01 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.882736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b2171-e97a-4b6b-8f1f-52f6a9e51b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.883262 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b2171-e97a-4b6b-8f1f-52f6a9e51b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:27.884156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b2171-e97a-4b6b-8f1f-52f6a9e51b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:27.886231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b2171-e97a-4b6b-8f1f-52f6a9e51b01 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:27.906667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 884/1769] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:27.908476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b2171-e97a-4b6b-8f1f-52f6a9e51b01 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:27.910975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0462bb3-0d5e-4dfa-ad4e-8fdb3326d693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:27.913641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0462bb3-0d5e-4dfa-ad4e-8fdb3326d693 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:27.920742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b2171-e97a-4b6b-8f1f-52f6a9e51b01 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:27.932736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb9811-c664-43a0-bf19-62f45a05ed38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:27.933477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb9811-c664-43a0-bf19-62f45a05ed38 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.933759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb9811-c664-43a0-bf19-62f45a05ed38 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.934201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb9811-c664-43a0-bf19-62f45a05ed38 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:27.935215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb9811-c664-43a0-bf19-62f45a05ed38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:27.936777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb9811-c664-43a0-bf19-62f45a05ed38 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:27.941141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 886/1770] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:27.943015 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e72af-8ae4-4a3b-9a63-bcc413cd32dc None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.943595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e72af-8ae4-4a3b-9a63-bcc413cd32dc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.945253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e72af-8ae4-4a3b-9a63-bcc413cd32dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:27.972118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 885/1771] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:27.972118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb9811-c664-43a0-bf19-62f45a05ed38 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:27.991223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb9811-c664-43a0-bf19-62f45a05ed38 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:27.997596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7820aff-c67a-4666-b79f-db2ce747ce03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:27.998861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820aff-c67a-4666-b79f-db2ce747ce03 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:27.999136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820aff-c67a-4666-b79f-db2ce747ce03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:27.999702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820aff-c67a-4666-b79f-db2ce747ce03 admin admin] PATH_INFO: `/v3/projects/3e61397de3e2424ca1199596d3c898ed/users/a619419679b443b28df925b44810c53a/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:28.049819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7820aff-c67a-4666-b79f-db2ce747ce03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e61397de3e2424ca1199596d3c898ed, user_id=a619419679b443b28df925b44810c53a, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:28.059293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7820aff-c67a-4666-b79f-db2ce747ce03 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:28.078473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-4e9e72af-8ae4-4a3b-9a63-bcc413cd32dc None None] MFA Rules not processed for user `569f41455cf948cfb39c90e6ffcc8fca`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:28.079745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4e9e72af-8ae4-4a3b-9a63-bcc413cd32dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:28.203472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 886/1772] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:27 2022] PUT /identity/v3/projects/3e61397de3e2424ca1199596d3c898ed/users/a619419679b443b28df925b44810c53a/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:28.204423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7820aff-c67a-4666-b79f-db2ce747ce03 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:28.229274 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:28.411350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 887/1773] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:28.421899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e72af-8ae4-4a3b-9a63-bcc413cd32dc None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:28.427811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7820aff-c67a-4666-b79f-db2ce747ce03 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:28.436246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d5cd6-91d2-4d04-a36b-aafb12fc8caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:28.436687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e72af-8ae4-4a3b-9a63-bcc413cd32dc None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:28.437116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d5cd6-91d2-4d04-a36b-aafb12fc8caf admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:28.437394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d5cd6-91d2-4d04-a36b-aafb12fc8caf admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:28.437667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d5cd6-91d2-4d04-a36b-aafb12fc8caf admin admin] PATH_INFO: `/v3/projects/0c652b3ae0534b9489f6374feff11a28/users/a9de993a28d54f5f87d72c90ad4d908a/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:28.439836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d5cd6-91d2-4d04-a36b-aafb12fc8caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c652b3ae0534b9489f6374feff11a28, user_id=a9de993a28d54f5f87d72c90ad4d908a, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:28.441521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef8ed9-233e-41d2-8a8b-ccd374af3475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:28.442329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef8ed9-233e-41d2-8a8b-ccd374af3475 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:28.442627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef8ed9-233e-41d2-8a8b-ccd374af3475 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:28.442777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d5cd6-91d2-4d04-a36b-aafb12fc8caf admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:28.442893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef8ed9-233e-41d2-8a8b-ccd374af3475 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:28.443558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef8ed9-233e-41d2-8a8b-ccd374af3475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:28.445998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef8ed9-233e-41d2-8a8b-ccd374af3475 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:28.454437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 887/1774] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:28 2022] PUT /identity/v3/projects/0c652b3ae0534b9489f6374feff11a28/users/a9de993a28d54f5f87d72c90ad4d908a/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:28.463408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d5cd6-91d2-4d04-a36b-aafb12fc8caf admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:28.464429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 888/1775] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:28.465878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbf141-026f-4f24-893c-517fe049e3d6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:28.466113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbf141-026f-4f24-893c-517fe049e3d6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:28.466447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbf141-026f-4f24-893c-517fe049e3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:28.468256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fb3d5cd6-91d2-4d04-a36b-aafb12fc8caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:28.502935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9dfbf141-026f-4f24-893c-517fe049e3d6 None None] MFA Rules not processed for user `a9de993a28d54f5f87d72c90ad4d908a`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:28.504241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9dfbf141-026f-4f24-893c-517fe049e3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:28.541826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d5cd6-91d2-4d04-a36b-aafb12fc8caf admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:28.546182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf29351-fe45-4b14-a73e-03d09cc06203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:28.546859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf29351-fe45-4b14-a73e-03d09cc06203 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:28.547168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf29351-fe45-4b14-a73e-03d09cc06203 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:28.547445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf29351-fe45-4b14-a73e-03d09cc06203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:28.591899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 889/1776] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:28.592766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbf141-026f-4f24-893c-517fe049e3d6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:28.642627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf29351-fe45-4b14-a73e-03d09cc06203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:28.644067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf29351-fe45-4b14-a73e-03d09cc06203 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:28.657629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 888/1777] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:28.667655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf29351-fe45-4b14-a73e-03d09cc06203 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:28.674206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbf141-026f-4f24-893c-517fe049e3d6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:28.682753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419071-b656-45d2-a559-657ed103afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:28.683479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419071-b656-45d2-a559-657ed103afe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:28.683756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419071-b656-45d2-a559-657ed103afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:28.684026 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419071-b656-45d2-a559-657ed103afe0 admin admin] PATH_INFO: `/v3/projects/3e61397de3e2424ca1199596d3c898ed/users/a619419679b443b28df925b44810c53a/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:28.685112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf29351-fe45-4b14-a73e-03d09cc06203 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:28.686365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7419071-b656-45d2-a559-657ed103afe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e61397de3e2424ca1199596d3c898ed, user_id=a619419679b443b28df925b44810c53a, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:28.689152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7419071-b656-45d2-a559-657ed103afe0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:28.693361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:28.694194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:28.695922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:28.696106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] PATH_INFO: `/v3/users/c7ff504bc358424c9271f629641b7209` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:28.698057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ff504bc358424c9271f629641b7209)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:28.700322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:28.702644 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 890/1778] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:28 2022] PUT /identity/v3/projects/3e61397de3e2424ca1199596d3c898ed/users/a619419679b443b28df925b44810c53a/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:28.703480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419071-b656-45d2-a559-657ed103afe0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:28.706884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f7419071-b656-45d2-a559-657ed103afe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:28.804125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419071-b656-45d2-a559-657ed103afe0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:28.808268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d63e24-7cc4-4208-a5b3-6859bfdf45af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:28.808895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d63e24-7cc4-4208-a5b3-6859bfdf45af service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:28.809174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d63e24-7cc4-4208-a5b3-6859bfdf45af service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:28.809446 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d63e24-7cc4-4208-a5b3-6859bfdf45af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:28.874983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d63e24-7cc4-4208-a5b3-6859bfdf45af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:28.876470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d63e24-7cc4-4208-a5b3-6859bfdf45af service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:28.888106 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 891/1779] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:28.889860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ceb9-baa4-428d-a849-e0ef3f2139c4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:28.890622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ceb9-baa4-428d-a849-e0ef3f2139c4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:28.891205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ceb9-baa4-428d-a849-e0ef3f2139c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:28.911251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ff504bc358424c9271f629641b7209'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:28.931227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dc90ceb9-baa4-428d-a849-e0ef3f2139c4 None None] MFA Rules not processed for user `a619419679b443b28df925b44810c53a`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:28.936004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dc90ceb9-baa4-428d-a849-e0ef3f2139c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:29.001069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ff504bc358424c9271f629641b7209'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:29.019537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 892/1780] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:29.020646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90ceb9-baa4-428d-a849-e0ef3f2139c4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:29.030473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ff504bc358424c9271f629641b7209'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:29.045243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90ceb9-baa4-428d-a849-e0ef3f2139c4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:29.045398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ff504bc358424c9271f629641b7209'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:29.045494 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:29.045991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:29.046094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5da877fd-417d-4d6a-a60b-fa0d5dae8fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ff504bc358424c9271f629641b7209'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:29.046269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:29.046541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:29.048732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:29.053594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 889/1781] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:28 2022] DELETE /identity/v3/users/c7ff504bc358424c9271f629641b7209 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:29.058055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb8665-195d-4c13-83f5-b4c25cd073ec None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:29.058342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb8665-195d-4c13-83f5-b4c25cd073ec None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:29.058620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb8665-195d-4c13-83f5-b4c25cd073ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:29.078917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-86eb8665-195d-4c13-83f5-b4c25cd073ec None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:29.080138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-86eb8665-195d-4c13-83f5-b4c25cd073ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:29.114345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:29.116748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:29.127830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 893/1782] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:29.164001 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 890/1783] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:00:29.182956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:29.182956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:29.197861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb8665-195d-4c13-83f5-b4c25cd073ec None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:29.199676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-86eb8665-195d-4c13-83f5-b4c25cd073ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:29.211020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:29.290319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fe4cc-6251-4af9-9eeb-45b565aa2d13 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:29.323407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb8665-195d-4c13-83f5-b4c25cd073ec None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:29.330027 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:29.330991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:29.331452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:29.331895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:29.334080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:29.371909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14994bfd-ec7f-4ba6-9fb4-f1dc988d4e7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:29.372824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14994bfd-ec7f-4ba6-9fb4-f1dc988d4e7d service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:29.373815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14994bfd-ec7f-4ba6-9fb4-f1dc988d4e7d service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:29.374131 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14994bfd-ec7f-4ba6-9fb4-f1dc988d4e7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:29.388686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14994bfd-ec7f-4ba6-9fb4-f1dc988d4e7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:29.390261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14994bfd-ec7f-4ba6-9fb4-f1dc988d4e7d service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:29.404499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 894/1784] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:29.485363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:29.487215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:29.503932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 891/1785] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:00:29.691292 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14994bfd-ec7f-4ba6-9fb4-f1dc988d4e7d service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:29.702821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14994bfd-ec7f-4ba6-9fb4-f1dc988d4e7d service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:29.708852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:29.710962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:29.711623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:29.712061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:29.715264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:29.755945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:29.757962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:29.785321 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0ce8f-d0f9-4baf-8945-d03d72c9ce31 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:29.792759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c46c0d-8a39-4bcd-900d-e97c0a766fc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:29.794722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c46c0d-8a39-4bcd-900d-e97c0a766fc1 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:29.795214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c46c0d-8a39-4bcd-900d-e97c0a766fc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:29.797146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c46c0d-8a39-4bcd-900d-e97c0a766fc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:29.816655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-57c46c0d-8a39-4bcd-900d-e97c0a766fc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:29.848033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:29.849927 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:29.864071 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 895/1786] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:29.923701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c46c0d-8a39-4bcd-900d-e97c0a766fc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:29.925314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c46c0d-8a39-4bcd-900d-e97c0a766fc1 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:29.940917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 892/1787] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:00:30.097219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:30.102759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:30.120773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de95942-1266-4049-93db-e3fdd2824b0a service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:30.127495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:30.128298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:30.128626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:30.128792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] PATH_INFO: `/v3/projects/9ed2adbf8e814dc2894fe0e8141edd4c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:30.130097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ed2adbf8e814dc2894fe0e8141edd4c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:30.132773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:30.309012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ed2adbf8e814dc2894fe0e8141edd4c', 'request_id': 'req-d1561f83-82aa-4954-9fb8-3d396045032b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:30.317834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ed2adbf8e814dc2894fe0e8141edd4c', 'request_id': 'req-d1561f83-82aa-4954-9fb8-3d396045032b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:30.423599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ed2adbf8e814dc2894fe0e8141edd4c', 'request_id': 'req-d1561f83-82aa-4954-9fb8-3d396045032b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:30.440495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] The token cache is being invalidate because project 9ed2adbf8e814dc2894fe0e8141edd4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:30.440819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:30.441564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1561f83-82aa-4954-9fb8-3d396045032b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:30.443226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 896/1788] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:30 2022] DELETE /identity/v3/projects/9ed2adbf8e814dc2894fe0e8141edd4c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:30.451111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117736a3-6bd1-4386-a5e5-88800af48f95 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:30.451541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117736a3-6bd1-4386-a5e5-88800af48f95 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:30.451966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117736a3-6bd1-4386-a5e5-88800af48f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:30.495290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-117736a3-6bd1-4386-a5e5-88800af48f95 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:30.497352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-117736a3-6bd1-4386-a5e5-88800af48f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:30.657347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 893/1789] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:30.657464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41365801-8326-4855-9ee6-db9b8705c652 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:30.657464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41365801-8326-4855-9ee6-db9b8705c652 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:30.657464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41365801-8326-4855-9ee6-db9b8705c652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:30.684934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-41365801-8326-4855-9ee6-db9b8705c652 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:30.691780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-41365801-8326-4855-9ee6-db9b8705c652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:30.704814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 897/1790] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:30.706232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117736a3-6bd1-4386-a5e5-88800af48f95 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:30.716664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:30.725432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117736a3-6bd1-4386-a5e5-88800af48f95 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:30.732069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c137fd-45b5-43b4-96b7-b3ca8afa8ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:30.733126 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c137fd-45b5-43b4-96b7-b3ca8afa8ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:30.733747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c137fd-45b5-43b4-96b7-b3ca8afa8ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:30.734380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c137fd-45b5-43b4-96b7-b3ca8afa8ff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:30.734839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c137fd-45b5-43b4-96b7-b3ca8afa8ff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:30.735271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c137fd-45b5-43b4-96b7-b3ca8afa8ff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:30.737566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c137fd-45b5-43b4-96b7-b3ca8afa8ff4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:30.754355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 894/1791] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:30.760256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41365801-8326-4855-9ee6-db9b8705c652 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:30.778078 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41365801-8326-4855-9ee6-db9b8705c652 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:30.794185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9238-d924-4d42-aba4-022db2c507a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:30.795302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9238-d924-4d42-aba4-022db2c507a7 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:30.795721 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9238-d924-4d42-aba4-022db2c507a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:30.796124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9238-d924-4d42-aba4-022db2c507a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:30.797132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b9238-d924-4d42-aba4-022db2c507a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:30.802277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b9238-d924-4d42-aba4-022db2c507a7 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:30.943119 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 898/1792] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:30 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:30.949489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c137fd-45b5-43b4-96b7-b3ca8afa8ff4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:30.961553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c137fd-45b5-43b4-96b7-b3ca8afa8ff4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:31.044285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93447deb-f72d-4310-abe6-ef66dab7ca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:31.045823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93447deb-f72d-4310-abe6-ef66dab7ca87 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:31.046345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93447deb-f72d-4310-abe6-ef66dab7ca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:31.046824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93447deb-f72d-4310-abe6-ef66dab7ca87 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:31.048305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93447deb-f72d-4310-abe6-ef66dab7ca87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:31.052012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93447deb-f72d-4310-abe6-ef66dab7ca87 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:31.195426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 895/1793] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:30 2022] POST /identity/v3/users => generated 288 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:31.200337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9238-d924-4d42-aba4-022db2c507a7 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:31.215445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9238-d924-4d42-aba4-022db2c507a7 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:31.220558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbd4e3-a1a1-4b22-bec3-75f5c013ec11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:31.221279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbd4e3-a1a1-4b22-bec3-75f5c013ec11 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:31.221670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbd4e3-a1a1-4b22-bec3-75f5c013ec11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:31.222072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbd4e3-a1a1-4b22-bec3-75f5c013ec11 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:31.222846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbd4e3-a1a1-4b22-bec3-75f5c013ec11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:31.224679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbd4e3-a1a1-4b22-bec3-75f5c013ec11 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:31.239168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 899/1794] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:31.243269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93447deb-f72d-4310-abe6-ef66dab7ca87 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:31.254656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93447deb-f72d-4310-abe6-ef66dab7ca87 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:31.263594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daabf6-9241-40f6-ada8-674430f18527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:31.264291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daabf6-9241-40f6-ada8-674430f18527 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:31.264600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daabf6-9241-40f6-ada8-674430f18527 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:31.264871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daabf6-9241-40f6-ada8-674430f18527 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:31.265400 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daabf6-9241-40f6-ada8-674430f18527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:31.267132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daabf6-9241-40f6-ada8-674430f18527 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:31.279325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 896/1795] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:31.283930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbd4e3-a1a1-4b22-bec3-75f5c013ec11 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:31.304517 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbd4e3-a1a1-4b22-bec3-75f5c013ec11 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:31.310088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b886499-81cc-4506-9dd5-f9cbec935676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:31.310769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b886499-81cc-4506-9dd5-f9cbec935676 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:31.311024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b886499-81cc-4506-9dd5-f9cbec935676 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:31.311321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b886499-81cc-4506-9dd5-f9cbec935676 admin admin] PATH_INFO: `/v3/projects/8d2885294fa1446782ac71b679d9d1e7/users/0d8f7df0786d4008a96dcae5c03ec8d5/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:31.329453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b886499-81cc-4506-9dd5-f9cbec935676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2885294fa1446782ac71b679d9d1e7, user_id=0d8f7df0786d4008a96dcae5c03ec8d5, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:31.332641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b886499-81cc-4506-9dd5-f9cbec935676 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:31.468587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 900/1796] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:31 2022] PUT /identity/v3/projects/8d2885294fa1446782ac71b679d9d1e7/users/0d8f7df0786d4008a96dcae5c03ec8d5/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:31.479799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec7a30-1179-4d01-a8e3-9d29feefde3b None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:31.480079 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec7a30-1179-4d01-a8e3-9d29feefde3b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:31.480362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec7a30-1179-4d01-a8e3-9d29feefde3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:31.513561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8cec7a30-1179-4d01-a8e3-9d29feefde3b None None] MFA Rules not processed for user `0d8f7df0786d4008a96dcae5c03ec8d5`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:31.516238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8cec7a30-1179-4d01-a8e3-9d29feefde3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:31.606025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 897/1797] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:31.610817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b886499-81cc-4506-9dd5-f9cbec935676 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:31.627576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:31.697544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b886499-81cc-4506-9dd5-f9cbec935676 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:31.714429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e1219-b2a8-4ab2-b074-0c709fab528a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:31.715794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e1219-b2a8-4ab2-b074-0c709fab528a admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:31.716612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e1219-b2a8-4ab2-b074-0c709fab528a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:31.717349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e1219-b2a8-4ab2-b074-0c709fab528a admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:31.719217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e1219-b2a8-4ab2-b074-0c709fab528a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:31.728896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e1219-b2a8-4ab2-b074-0c709fab528a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:31.732667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:31.892600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 901/1798] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:31 2022] POST /identity/v3/projects => generated 375 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:31.894379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec7a30-1179-4d01-a8e3-9d29feefde3b None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:31.931216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec7a30-1179-4d01-a8e3-9d29feefde3b None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:32.055571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ac946-b1e6-4ead-9619-eed02689a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:32.056231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ac946-b1e6-4ead-9619-eed02689a31d admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:32.056508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ac946-b1e6-4ead-9619-eed02689a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:32.056785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ac946-b1e6-4ead-9619-eed02689a31d admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:32.057434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ac946-b1e6-4ead-9619-eed02689a31d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:32.059611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ac946-b1e6-4ead-9619-eed02689a31d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:32.185387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 898/1799] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:31 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:32.190038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e1219-b2a8-4ab2-b074-0c709fab528a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:32.203353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e1219-b2a8-4ab2-b074-0c709fab528a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:32.209058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8569c8-7f12-4cfb-81e4-967c30059692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:32.209974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8569c8-7f12-4cfb-81e4-967c30059692 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:32.210402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8569c8-7f12-4cfb-81e4-967c30059692 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:32.211004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8569c8-7f12-4cfb-81e4-967c30059692 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:32.211805 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8569c8-7f12-4cfb-81e4-967c30059692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:32.213706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8569c8-7f12-4cfb-81e4-967c30059692 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:32.231130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 902/1800] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:32.231930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ac946-b1e6-4ead-9619-eed02689a31d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:32.243851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ac946-b1e6-4ead-9619-eed02689a31d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:32.248814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d7efd-f4f4-46de-959d-e5b54e1578e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:32.249476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d7efd-f4f4-46de-959d-e5b54e1578e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:32.249982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d7efd-f4f4-46de-959d-e5b54e1578e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:32.250103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d7efd-f4f4-46de-959d-e5b54e1578e1 admin admin] PATH_INFO: `/v3/projects/deb0c73a1f7942a99bfbbb6d49b3ba05/users/7ca738fea6a0471d9eecc084a322598d/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:32.266242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d7efd-f4f4-46de-959d-e5b54e1578e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb0c73a1f7942a99bfbbb6d49b3ba05, user_id=7ca738fea6a0471d9eecc084a322598d, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:32.268921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d7efd-f4f4-46de-959d-e5b54e1578e1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:32.343186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 899/1801] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:32 2022] PUT /identity/v3/projects/deb0c73a1f7942a99bfbbb6d49b3ba05/users/7ca738fea6a0471d9eecc084a322598d/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:32.347429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8569c8-7f12-4cfb-81e4-967c30059692 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:32.358565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8569c8-7f12-4cfb-81e4-967c30059692 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:32.423572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682711ae-1e8a-4618-b727-c3fd6c53bb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:32.424192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682711ae-1e8a-4618-b727-c3fd6c53bb4e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:32.424631 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682711ae-1e8a-4618-b727-c3fd6c53bb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:32.424934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682711ae-1e8a-4618-b727-c3fd6c53bb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:32.425704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682711ae-1e8a-4618-b727-c3fd6c53bb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:32.428758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682711ae-1e8a-4618-b727-c3fd6c53bb4e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:32.442160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 903/1802] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:32.446488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d7efd-f4f4-46de-959d-e5b54e1578e1 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:32.465632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d7efd-f4f4-46de-959d-e5b54e1578e1 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:32.471654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4753539-d609-4782-a551-258f32ee418c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:32.472326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4753539-d609-4782-a551-258f32ee418c admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:32.472705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4753539-d609-4782-a551-258f32ee418c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:32.472980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4753539-d609-4782-a551-258f32ee418c admin admin] PATH_INFO: `/v3/projects/deb0c73a1f7942a99bfbbb6d49b3ba05/users/7ca738fea6a0471d9eecc084a322598d/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:32.475488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4753539-d609-4782-a551-258f32ee418c admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb0c73a1f7942a99bfbbb6d49b3ba05, user_id=7ca738fea6a0471d9eecc084a322598d, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:32.478187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4753539-d609-4782-a551-258f32ee418c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:32.491764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 900/1803] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:32 2022] PUT /identity/v3/projects/deb0c73a1f7942a99bfbbb6d49b3ba05/users/7ca738fea6a0471d9eecc084a322598d/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:32.504506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc66b-4a9e-4aba-bf70-8d16a9410daa None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:32.504912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc66b-4a9e-4aba-bf70-8d16a9410daa None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:32.505184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc66b-4a9e-4aba-bf70-8d16a9410daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:32.535811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-78afc66b-4a9e-4aba-bf70-8d16a9410daa None None] MFA Rules not processed for user `7ca738fea6a0471d9eecc084a322598d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:32.537012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-78afc66b-4a9e-4aba-bf70-8d16a9410daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:32.614178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 904/1804] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:32.819680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4753539-d609-4782-a551-258f32ee418c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:32.819680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b4753539-d609-4782-a551-258f32ee418c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:32.922229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4753539-d609-4782-a551-258f32ee418c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:32.926979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6589c64-5841-4517-910f-cae2a8e8d46a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:32.927614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6589c64-5841-4517-910f-cae2a8e8d46a service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:32.927967 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6589c64-5841-4517-910f-cae2a8e8d46a service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:32.928244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6589c64-5841-4517-910f-cae2a8e8d46a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:33.009655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6589c64-5841-4517-910f-cae2a8e8d46a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:33.009966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6589c64-5841-4517-910f-cae2a8e8d46a service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:33.020679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 901/1805] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:33.140396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:33.526617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:34.231264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:34.360479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb18ef-bd94-4e77-b0ad-48bf89fabc07 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:34.361449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb18ef-bd94-4e77-b0ad-48bf89fabc07 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:34.361900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb18ef-bd94-4e77-b0ad-48bf89fabc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:34.522998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-18eb18ef-bd94-4e77-b0ad-48bf89fabc07 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:34.524242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-18eb18ef-bd94-4e77-b0ad-48bf89fabc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:34.635571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 905/1806] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:00:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:00:35.114286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6589c64-5841-4517-910f-cae2a8e8d46a service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:35.116851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b6589c64-5841-4517-910f-cae2a8e8d46a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:35.242539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:35.259137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6589c64-5841-4517-910f-cae2a8e8d46a service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:35.265483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15649ca0-7774-41e9-9b5d-30569ef3d496 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:35.266175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649ca0-7774-41e9-9b5d-30569ef3d496 service swift] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:35.267057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649ca0-7774-41e9-9b5d-30569ef3d496 service swift] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:35.267239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649ca0-7774-41e9-9b5d-30569ef3d496 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:35.279704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649ca0-7774-41e9-9b5d-30569ef3d496 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:35.281424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649ca0-7774-41e9-9b5d-30569ef3d496 service swift] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:35.291322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 902/1807] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:00:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:00:35.953013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:39.195911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:40.263653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:40.938452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb18ef-bd94-4e77-b0ad-48bf89fabc07 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:40.940115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-18eb18ef-bd94-4e77-b0ad-48bf89fabc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:41.056784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb18ef-bd94-4e77-b0ad-48bf89fabc07 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:41.064275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:41.065169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:41.065520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:41.065949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] PATH_INFO: `/v3/users/25b6a59596144263a9bd0b798886b0c9` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:41.067638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b6a59596144263a9bd0b798886b0c9)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:41.070362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:41.348848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b6a59596144263a9bd0b798886b0c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.594938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b6a59596144263a9bd0b798886b0c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.621655 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b6a59596144263a9bd0b798886b0c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.638559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b6a59596144263a9bd0b798886b0c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.639990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b6a59596144263a9bd0b798886b0c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:41.679990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 906/1808] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:40 2022] DELETE /identity/v3/users/25b6a59596144263a9bd0b798886b0c9 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:41.682902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d0335-a058-4178-a2e5-df94e451cf22 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:41.683596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d0335-a058-4178-a2e5-df94e451cf22 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:41.683596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d0335-a058-4178-a2e5-df94e451cf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:41.716780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b92d0335-a058-4178-a2e5-df94e451cf22 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:41.718236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b92d0335-a058-4178-a2e5-df94e451cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:41.803414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 903/1809] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:41.813070 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:41.814576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:41.985955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc02790-bb72-405f-9c62-6cb96d6600ce admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:41.985955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b46aee-22d8-4b3a-a77e-0127bb2298c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:41.985955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b46aee-22d8-4b3a-a77e-0127bb2298c5 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:41.985955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b46aee-22d8-4b3a-a77e-0127bb2298c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:41.986815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b46aee-22d8-4b3a-a77e-0127bb2298c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:42.002479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b46aee-22d8-4b3a-a77e-0127bb2298c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:42.004291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b46aee-22d8-4b3a-a77e-0127bb2298c5 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:42.022411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 907/1810] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:42.934201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d0335-a058-4178-a2e5-df94e451cf22 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:42.936424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b92d0335-a058-4178-a2e5-df94e451cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:42.948649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d0335-a058-4178-a2e5-df94e451cf22 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:42.959499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:42.960154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:42.960430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:42.960750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] PATH_INFO: `/v3/projects/2376b9d6148d496bb88cdba4cf2a77a9` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:42.963772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2376b9d6148d496bb88cdba4cf2a77a9)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:42.965971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:43.105951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2376b9d6148d496bb88cdba4cf2a77a9', 'request_id': 'req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:43.114459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2376b9d6148d496bb88cdba4cf2a77a9', 'request_id': 'req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:43.422695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2376b9d6148d496bb88cdba4cf2a77a9', 'request_id': 'req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:43.448751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] The token cache is being invalidate because project 2376b9d6148d496bb88cdba4cf2a77a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:43.449128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:43.450188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:43.455822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 904/1811] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:42 2022] DELETE /identity/v3/projects/2376b9d6148d496bb88cdba4cf2a77a9 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:43.459728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b46aee-22d8-4b3a-a77e-0127bb2298c5 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:43.471429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b46aee-22d8-4b3a-a77e-0127bb2298c5 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:43.545382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:43.546049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:43.546332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:43.546635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] PATH_INFO: `/v3/users/a619419679b443b28df925b44810c53a` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:43.547800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a619419679b443b28df925b44810c53a)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:43.549761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:43.820507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a619419679b443b28df925b44810c53a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:44.028460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a619419679b443b28df925b44810c53a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:44.046298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a619419679b443b28df925b44810c53a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:44.054096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a619419679b443b28df925b44810c53a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:44.054842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a619419679b443b28df925b44810c53a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:44.057964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:44.063254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 908/1812] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:43 2022] DELETE /identity/v3/users/a619419679b443b28df925b44810c53a => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:44.774909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:44.774909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:44.794676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3468c7-ea5d-48a1-9625-0ed8b28214ed admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:44.885424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:44.903382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:44.904257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:44.904537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:44.904812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] PATH_INFO: `/v3/projects/3e61397de3e2424ca1199596d3c898ed` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:44.906120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e61397de3e2424ca1199596d3c898ed)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:44.908254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:44.999721 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:45.003438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:45.072055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a684926-d18c-48b7-a482-7981aa9fce7d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:45.082596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eff7d-4652-45c9-82b9-c0e8b393491c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:45.083495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eff7d-4652-45c9-82b9-c0e8b393491c service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:45.083775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eff7d-4652-45c9-82b9-c0e8b393491c service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:45.084238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eff7d-4652-45c9-82b9-c0e8b393491c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:45.086371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5f3eff7d-4652-45c9-82b9-c0e8b393491c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:45.194309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3eff7d-4652-45c9-82b9-c0e8b393491c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:45.195891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3eff7d-4652-45c9-82b9-c0e8b393491c service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:45.199208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e61397de3e2424ca1199596d3c898ed', 'request_id': 'req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:45.207678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 909/1813] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:45.211030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e61397de3e2424ca1199596d3c898ed', 'request_id': 'req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:45.387989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e61397de3e2424ca1199596d3c898ed', 'request_id': 'req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:45.400345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] The token cache is being invalidate because project 3e61397de3e2424ca1199596d3c898ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:45.405625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:45.405625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-93aa7797-c5d4-4729-bbb9-21f6b448f7f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:45.405625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 905/1814] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:44 2022] DELETE /identity/v3/projects/3e61397de3e2424ca1199596d3c898ed => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:45.407913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74326f8b-8445-4430-92fe-bd4dad171ef5 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:45.408198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74326f8b-8445-4430-92fe-bd4dad171ef5 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:45.408471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74326f8b-8445-4430-92fe-bd4dad171ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:45.431080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-74326f8b-8445-4430-92fe-bd4dad171ef5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:45.432313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-74326f8b-8445-4430-92fe-bd4dad171ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:45.541942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 910/1815] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:45.553588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad58007-2169-45ea-a99a-6605c4345a40 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:45.553994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad58007-2169-45ea-a99a-6605c4345a40 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:45.554326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad58007-2169-45ea-a99a-6605c4345a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:45.579200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8ad58007-2169-45ea-a99a-6605c4345a40 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:45.580498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8ad58007-2169-45ea-a99a-6605c4345a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:45.599564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 906/1816] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:45.606916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74326f8b-8445-4430-92fe-bd4dad171ef5 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:45.629171 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74326f8b-8445-4430-92fe-bd4dad171ef5 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:45.634188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1112de-1605-42e8-9f85-3000211ac99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:45.634825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1112de-1605-42e8-9f85-3000211ac99e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:45.635160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1112de-1605-42e8-9f85-3000211ac99e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:45.635435 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1112de-1605-42e8-9f85-3000211ac99e admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:45.636112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1112de-1605-42e8-9f85-3000211ac99e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:45.636419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1112de-1605-42e8-9f85-3000211ac99e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:45.638923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1112de-1605-42e8-9f85-3000211ac99e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:45.665130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 911/1817] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:00:45.670920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad58007-2169-45ea-a99a-6605c4345a40 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:45.690391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad58007-2169-45ea-a99a-6605c4345a40 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:45.717419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30c17b-fa1a-4a1c-b862-172630463704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:45.719990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30c17b-fa1a-4a1c-b862-172630463704 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:45.720340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30c17b-fa1a-4a1c-b862-172630463704 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:45.720689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30c17b-fa1a-4a1c-b862-172630463704 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:45.721412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30c17b-fa1a-4a1c-b862-172630463704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:45.725887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30c17b-fa1a-4a1c-b862-172630463704 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:46.222180 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 907/1818] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:45 2022] POST /identity/v3/projects => generated 389 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:46.229008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1112de-1605-42e8-9f85-3000211ac99e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:46.245679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1112de-1605-42e8-9f85-3000211ac99e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:46.337865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d24ad2-4c47-4bde-9b2e-ddac57e5575a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:46.338482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d24ad2-4c47-4bde-9b2e-ddac57e5575a admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:46.338764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d24ad2-4c47-4bde-9b2e-ddac57e5575a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:46.339215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d24ad2-4c47-4bde-9b2e-ddac57e5575a admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:46.340008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d24ad2-4c47-4bde-9b2e-ddac57e5575a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:46.342161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d24ad2-4c47-4bde-9b2e-ddac57e5575a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:46.720137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5894e59-89e3-4888-9397-26c99062c5c0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:46.720422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5894e59-89e3-4888-9397-26c99062c5c0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:46.720711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5894e59-89e3-4888-9397-26c99062c5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:46.742876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e5894e59-89e3-4888-9397-26c99062c5c0 None None] MFA Rules not processed for user `f50ee57e2418417cbe68915ca3f00f76`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:46.744136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e5894e59-89e3-4888-9397-26c99062c5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:46.804702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 908/1819] 10.222.0.67 () {58 vars in 1015 bytes} [Mon Aug 8 17:00:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:46.816579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 912/1820] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:46 2022] POST /identity/v3/users => generated 295 bytes in 588 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:46.822028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5894e59-89e3-4888-9397-26c99062c5c0 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:46.834420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5894e59-89e3-4888-9397-26c99062c5c0 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:46.839462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb967a-11eb-4c80-bede-a132a9b9a0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:46.840183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb967a-11eb-4c80-bede-a132a9b9a0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:46.840573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb967a-11eb-4c80-bede-a132a9b9a0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:46.840860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb967a-11eb-4c80-bede-a132a9b9a0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:46.841648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb967a-11eb-4c80-bede-a132a9b9a0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:46.843616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb967a-11eb-4c80-bede-a132a9b9a0b8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:46.854778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 909/1821] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:46.859470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d24ad2-4c47-4bde-9b2e-ddac57e5575a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:46.869897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d24ad2-4c47-4bde-9b2e-ddac57e5575a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:46.874661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bdb6b-5593-435a-94fb-97936c8dc7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:46.875265 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bdb6b-5593-435a-94fb-97936c8dc7fa admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:46.875549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bdb6b-5593-435a-94fb-97936c8dc7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:46.875851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bdb6b-5593-435a-94fb-97936c8dc7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:46.876505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bdb6b-5593-435a-94fb-97936c8dc7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:46.878513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bdb6b-5593-435a-94fb-97936c8dc7fa admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:46.890997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 913/1822] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:46.895406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb967a-11eb-4c80-bede-a132a9b9a0b8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:46.907494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb967a-11eb-4c80-bede-a132a9b9a0b8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:46.913994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dd935-879b-4fa7-b0cc-eea3cd61ee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:46.914811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dd935-879b-4fa7-b0cc-eea3cd61ee4c admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:46.915129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dd935-879b-4fa7-b0cc-eea3cd61ee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:46.915442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dd935-879b-4fa7-b0cc-eea3cd61ee4c admin admin] PATH_INFO: `/v3/projects/1408c73523bb463c88702f63fb3d09de/users/e0ebcb3a0f4b48f28f74081da9c36180/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:46.943774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6dd935-879b-4fa7-b0cc-eea3cd61ee4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1408c73523bb463c88702f63fb3d09de, user_id=e0ebcb3a0f4b48f28f74081da9c36180, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:46.947485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6dd935-879b-4fa7-b0cc-eea3cd61ee4c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:47.089781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bdb6b-5593-435a-94fb-97936c8dc7fa admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:47.091786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2b7bdb6b-5593-435a-94fb-97936c8dc7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:47.117134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bdb6b-5593-435a-94fb-97936c8dc7fa admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:47.117314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 910/1823] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:46 2022] PUT /identity/v3/projects/1408c73523bb463c88702f63fb3d09de/users/e0ebcb3a0f4b48f28f74081da9c36180/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:47.117314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dd935-879b-4fa7-b0cc-eea3cd61ee4c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:47.213931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507cede-3c06-4efc-a456-38080cc8b07a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:47.213931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507cede-3c06-4efc-a456-38080cc8b07a service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:47.213931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507cede-3c06-4efc-a456-38080cc8b07a service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:47.213931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507cede-3c06-4efc-a456-38080cc8b07a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:47.218429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507cede-3c06-4efc-a456-38080cc8b07a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:47.219906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507cede-3c06-4efc-a456-38080cc8b07a service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:47.241224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 914/1824] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:00:47.245973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efed7f-9c18-4662-bc4a-83a91c3f51ba None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:47.246530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efed7f-9c18-4662-bc4a-83a91c3f51ba None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:47.246716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dd935-879b-4fa7-b0cc-eea3cd61ee4c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:47.247528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efed7f-9c18-4662-bc4a-83a91c3f51ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:47.251678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381637d3-e6cb-4fd6-89d9-d17bdd81ef41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:47.252525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381637d3-e6cb-4fd6-89d9-d17bdd81ef41 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:47.253136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381637d3-e6cb-4fd6-89d9-d17bdd81ef41 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:47.253668 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381637d3-e6cb-4fd6-89d9-d17bdd81ef41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:47.265418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381637d3-e6cb-4fd6-89d9-d17bdd81ef41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:47.265418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381637d3-e6cb-4fd6-89d9-d17bdd81ef41 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:47.281645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 911/1825] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:47.282745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381637d3-e6cb-4fd6-89d9-d17bdd81ef41 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:47.283921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-381637d3-e6cb-4fd6-89d9-d17bdd81ef41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:47.301418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381637d3-e6cb-4fd6-89d9-d17bdd81ef41 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:47.306204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2fc9a-c1e9-41b7-9733-c68e04fb0c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:47.307045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2fc9a-c1e9-41b7-9733-c68e04fb0c6a service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:47.307525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2fc9a-c1e9-41b7-9733-c68e04fb0c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:47.308001 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2fc9a-c1e9-41b7-9733-c68e04fb0c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:47.308361 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-e8efed7f-9c18-4662-bc4a-83a91c3f51ba None None] MFA Rules not processed for user `e0ebcb3a0f4b48f28f74081da9c36180`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:47.309913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e8efed7f-9c18-4662-bc4a-83a91c3f51ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:47.321473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2fc9a-c1e9-41b7-9733-c68e04fb0c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:47.323279 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2fc9a-c1e9-41b7-9733-c68e04fb0c6a service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:47.333217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 912/1826] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:47.334362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2fc9a-c1e9-41b7-9733-c68e04fb0c6a service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:47.344293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2fc9a-c1e9-41b7-9733-c68e04fb0c6a service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:47.348579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a6a61-9ac2-4dc5-b83c-63f9bfea27fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:47.349733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a6a61-9ac2-4dc5-b83c-63f9bfea27fc service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:47.350051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a6a61-9ac2-4dc5-b83c-63f9bfea27fc service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:47.350325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a6a61-9ac2-4dc5-b83c-63f9bfea27fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:47.358562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a6a61-9ac2-4dc5-b83c-63f9bfea27fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:47.367610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a6a61-9ac2-4dc5-b83c-63f9bfea27fc service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:47.372422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 913/1827] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:47.411087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 915/1828] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:47.432925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a6a61-9ac2-4dc5-b83c-63f9bfea27fc service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:47.433757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efed7f-9c18-4662-bc4a-83a91c3f51ba None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:47.438212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e8efed7f-9c18-4662-bc4a-83a91c3f51ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:47.546258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efed7f-9c18-4662-bc4a-83a91c3f51ba None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:47.558811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a6a61-9ac2-4dc5-b83c-63f9bfea27fc service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:47.558999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:47.558999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:47.558999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:47.560130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858bece-0ecb-4a2f-a2c5-479645f24ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:47.560309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] PATH_INFO: `/v3/users/6f2f1e72bab542648a9142d974ec5d03` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:47.561214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858bece-0ecb-4a2f-a2c5-479645f24ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:47.561858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858bece-0ecb-4a2f-a2c5-479645f24ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:47.562335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858bece-0ecb-4a2f-a2c5-479645f24ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:47.563219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858bece-0ecb-4a2f-a2c5-479645f24ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:47.563471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f2f1e72bab542648a9142d974ec5d03)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:47.565602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858bece-0ecb-4a2f-a2c5-479645f24ff8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:47.565735 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:47.724575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 914/1829] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:47 2022] POST /identity/v3/projects => generated 389 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:47.731599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858bece-0ecb-4a2f-a2c5-479645f24ff8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:47.806911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858bece-0ecb-4a2f-a2c5-479645f24ff8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:47.811862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf71699-3539-44b2-959b-988c74d55a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:47.812489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf71699-3539-44b2-959b-988c74d55a02 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:47.812772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf71699-3539-44b2-959b-988c74d55a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:47.813045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf71699-3539-44b2-959b-988c74d55a02 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:47.814002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf71699-3539-44b2-959b-988c74d55a02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:47.824458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf71699-3539-44b2-959b-988c74d55a02 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:47.839536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f2f1e72bab542648a9142d974ec5d03'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:47.995125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f2f1e72bab542648a9142d974ec5d03'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:48.016550 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f2f1e72bab542648a9142d974ec5d03'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:48.027527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f2f1e72bab542648a9142d974ec5d03'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:48.028751 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-60bd7e67-a7a1-456f-855a-9fb9c689ec8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f2f1e72bab542648a9142d974ec5d03'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:48.039959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 916/1830] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:47 2022] DELETE /identity/v3/users/6f2f1e72bab542648a9142d974ec5d03 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:48.067069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800e819-fb12-4930-89ef-a350afece246 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.067435 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800e819-fb12-4930-89ef-a350afece246 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.068235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800e819-fb12-4930-89ef-a350afece246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.088647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c800e819-fb12-4930-89ef-a350afece246 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:48.089915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c800e819-fb12-4930-89ef-a350afece246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:48.105437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 915/1831] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:47 2022] POST /identity/v3/users => generated 301 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:48.106645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf71699-3539-44b2-959b-988c74d55a02 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:48.111526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3bf71699-3539-44b2-959b-988c74d55a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:48.195008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf71699-3539-44b2-959b-988c74d55a02 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:48.195452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 917/1832] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:00:48.195550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3477b81-b9f7-43bc-af06-da8c0fc65251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:48.195550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3477b81-b9f7-43bc-af06-da8c0fc65251 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.195550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3477b81-b9f7-43bc-af06-da8c0fc65251 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.195550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3477b81-b9f7-43bc-af06-da8c0fc65251 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.195881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3477b81-b9f7-43bc-af06-da8c0fc65251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:48.195881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3477b81-b9f7-43bc-af06-da8c0fc65251 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:48.195881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 916/1833] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:00:48.200012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800e819-fb12-4930-89ef-a350afece246 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:48.213690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800e819-fb12-4930-89ef-a350afece246 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:48.231840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc6406-1f09-4c78-9a51-282f992349da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:48.233504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc6406-1f09-4c78-9a51-282f992349da admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.233935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3477b81-b9f7-43bc-af06-da8c0fc65251 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:48.236477 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc6406-1f09-4c78-9a51-282f992349da admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.237024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc6406-1f09-4c78-9a51-282f992349da admin admin] PATH_INFO: `/v3/projects/4d5dd415b85d4d1392f43248a7c9b24b/users/f380db24d3ad43309f8eb716b7e3c976/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.237986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b3477b81-b9f7-43bc-af06-da8c0fc65251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:48.264985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc6406-1f09-4c78-9a51-282f992349da admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5dd415b85d4d1392f43248a7c9b24b, user_id=f380db24d3ad43309f8eb716b7e3c976, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:48.267986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc6406-1f09-4c78-9a51-282f992349da admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:48.322550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3477b81-b9f7-43bc-af06-da8c0fc65251 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:48.327416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a6fe1-929e-4dd0-8a01-7aa87383d045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:48.328038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a6fe1-929e-4dd0-8a01-7aa87383d045 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.328486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a6fe1-929e-4dd0-8a01-7aa87383d045 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.328951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a6fe1-929e-4dd0-8a01-7aa87383d045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.340615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a6fe1-929e-4dd0-8a01-7aa87383d045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:48.350443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a6fe1-929e-4dd0-8a01-7aa87383d045 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:48.392209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 917/1834] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:48.515013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 918/1835] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:48 2022] PUT /identity/v3/projects/4d5dd415b85d4d1392f43248a7c9b24b/users/f380db24d3ad43309f8eb716b7e3c976/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:48.515225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a6fe1-929e-4dd0-8a01-7aa87383d045 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:48.515225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a6fe1-929e-4dd0-8a01-7aa87383d045 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:48.672479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c78e3-8632-4ae3-999a-b097efe4b10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:48.673123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c78e3-8632-4ae3-999a-b097efe4b10a admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.673387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c78e3-8632-4ae3-999a-b097efe4b10a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.673659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c78e3-8632-4ae3-999a-b097efe4b10a admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.674362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c78e3-8632-4ae3-999a-b097efe4b10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:48.676419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c78e3-8632-4ae3-999a-b097efe4b10a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:48.698442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 918/1836] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:48.698679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc6406-1f09-4c78-9a51-282f992349da admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:48.707826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc6406-1f09-4c78-9a51-282f992349da admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:48.714402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638dd0b-cbab-4fcb-8381-bb13f7aa4cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:48.715430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638dd0b-cbab-4fcb-8381-bb13f7aa4cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.715887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638dd0b-cbab-4fcb-8381-bb13f7aa4cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.716383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638dd0b-cbab-4fcb-8381-bb13f7aa4cb7 admin admin] PATH_INFO: `/v3/projects/4d5dd415b85d4d1392f43248a7c9b24b/users/f380db24d3ad43309f8eb716b7e3c976/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.719096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638dd0b-cbab-4fcb-8381-bb13f7aa4cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5dd415b85d4d1392f43248a7c9b24b, user_id=f380db24d3ad43309f8eb716b7e3c976, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:48.722754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638dd0b-cbab-4fcb-8381-bb13f7aa4cb7 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:48.734362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 919/1837] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:48 2022] PUT /identity/v3/projects/4d5dd415b85d4d1392f43248a7c9b24b/users/f380db24d3ad43309f8eb716b7e3c976/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:48.745808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2dd0-304d-4ae8-822f-f6cef56dfe1e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.745808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2dd0-304d-4ae8-822f-f6cef56dfe1e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.746142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2dd0-304d-4ae8-822f-f6cef56dfe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.778198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8cfe2dd0-304d-4ae8-822f-f6cef56dfe1e None None] MFA Rules not processed for user `f380db24d3ad43309f8eb716b7e3c976`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:48.779360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8cfe2dd0-304d-4ae8-822f-f6cef56dfe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:48.821880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28daf14-ac8b-4f70-b336-5f0e43f58b37 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.822179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28daf14-ac8b-4f70-b336-5f0e43f58b37 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.822493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28daf14-ac8b-4f70-b336-5f0e43f58b37 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.824247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 920/1838] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 17:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 17:00:48.834277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19476a-57ed-4d68-a837-be076440534e None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.834981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19476a-57ed-4d68-a837-be076440534e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.834981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19476a-57ed-4d68-a837-be076440534e None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.838312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 921/1839] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 17:00:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:00:48.846452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115062c-90be-4eaf-8542-40920e17952f None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.847341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115062c-90be-4eaf-8542-40920e17952f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.848019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115062c-90be-4eaf-8542-40920e17952f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.887045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 919/1840] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:48 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:48.887045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57c661-206c-4e1a-b525-d6eb1a4f3b97 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:48.887045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57c661-206c-4e1a-b525-d6eb1a4f3b97 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:48.887045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57c661-206c-4e1a-b525-d6eb1a4f3b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:48.887513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-6115062c-90be-4eaf-8542-40920e17952f None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:48.887513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6115062c-90be-4eaf-8542-40920e17952f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:48.911835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-0e57c661-206c-4e1a-b525-d6eb1a4f3b97 None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:48.917488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0e57c661-206c-4e1a-b525-d6eb1a4f3b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:49.011838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 920/1841] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 17:00:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:49.013361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57c661-206c-4e1a-b525-d6eb1a4f3b97 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:49.019186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0e57c661-206c-4e1a-b525-d6eb1a4f3b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:49.048237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 922/1842] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 17:00:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:49.049254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115062c-90be-4eaf-8542-40920e17952f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:49.051165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6115062c-90be-4eaf-8542-40920e17952f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:49.093895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:49.094294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57c661-206c-4e1a-b525-d6eb1a4f3b97 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:49.103376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a8f1f-d428-499f-8dc0-6592cdf311d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:49.103376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a8f1f-d428-499f-8dc0-6592cdf311d8 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:49.103376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a8f1f-d428-499f-8dc0-6592cdf311d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:49.103376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a8f1f-d428-499f-8dc0-6592cdf311d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:49.141149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a8f1f-d428-499f-8dc0-6592cdf311d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:49.143192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a8f1f-d428-499f-8dc0-6592cdf311d8 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:49.155030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 921/1843] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:49 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:49.156160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a8f1f-d428-499f-8dc0-6592cdf311d8 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:49.157741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7f3a8f1f-d428-499f-8dc0-6592cdf311d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:49.168811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115062c-90be-4eaf-8542-40920e17952f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:49.178198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a8f1f-d428-499f-8dc0-6592cdf311d8 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:49.180765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:49.181381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:49.181770 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:49.182215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] PATH_INFO: `/v3/users/158fac32de4148279458af2ab4a7ed85` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:49.184844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=158fac32de4148279458af2ab4a7ed85)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:49.185720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fe28a-da32-49c4-9dbe-803e0241aa57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:49.189133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fe28a-da32-49c4-9dbe-803e0241aa57 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:49.189133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fe28a-da32-49c4-9dbe-803e0241aa57 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:49.189133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fe28a-da32-49c4-9dbe-803e0241aa57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:49.189559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:49.203045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fe28a-da32-49c4-9dbe-803e0241aa57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:49.204760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fe28a-da32-49c4-9dbe-803e0241aa57 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:49.233608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 922/1844] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:49.234677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fe28a-da32-49c4-9dbe-803e0241aa57 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:49.238341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ba6fe28a-da32-49c4-9dbe-803e0241aa57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:49.280465 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fe28a-da32-49c4-9dbe-803e0241aa57 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:49.298795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e35e8-6a0a-43a5-b618-00297c4aa4e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:49.306266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e35e8-6a0a-43a5-b618-00297c4aa4e2 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:49.306266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e35e8-6a0a-43a5-b618-00297c4aa4e2 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:49.306266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e35e8-6a0a-43a5-b618-00297c4aa4e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:49.318179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e35e8-6a0a-43a5-b618-00297c4aa4e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:49.320342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e35e8-6a0a-43a5-b618-00297c4aa4e2 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:49.362299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 923/1845] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:49.396332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e35e8-6a0a-43a5-b618-00297c4aa4e2 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:49.408550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e35e8-6a0a-43a5-b618-00297c4aa4e2 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:49.414947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:49.415712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:49.416058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:49.416401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] PATH_INFO: `/v3/users/e0ebcb3a0f4b48f28f74081da9c36180` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:49.418001 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ebcb3a0f4b48f28f74081da9c36180)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:49.421025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:49.532154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158fac32de4148279458af2ab4a7ed85'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:49.645280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158fac32de4148279458af2ab4a7ed85'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:49.658377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158fac32de4148279458af2ab4a7ed85'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:49.672374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158fac32de4148279458af2ab4a7ed85'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:49.673138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158fac32de4148279458af2ab4a7ed85'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:49.681413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 923/1846] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:49 2022] DELETE /identity/v3/users/158fac32de4148279458af2ab4a7ed85 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:49.683062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:49.684454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:49.888249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ebcb3a0f4b48f28f74081da9c36180'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:49.888938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455375d-404a-45dc-8714-20bb94342592 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:49.888938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:49.888938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:49.888938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:49.889493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] PATH_INFO: `/v3/projects/106c6c134f7b418699e3097adb022604` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:49.889493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=106c6c134f7b418699e3097adb022604)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:49.889493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:50.163917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:50.204959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ebcb3a0f4b48f28f74081da9c36180'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:50.213875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ebcb3a0f4b48f28f74081da9c36180'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:50.226353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106c6c134f7b418699e3097adb022604', 'request_id': 'req-deb93270-9ebc-4ce1-9120-a28640f724cc'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:50.231622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ebcb3a0f4b48f28f74081da9c36180'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:50.232366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-37720ad8-a801-4331-bfbb-029afd821325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ebcb3a0f4b48f28f74081da9c36180'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:50.233044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106c6c134f7b418699e3097adb022604', 'request_id': 'req-deb93270-9ebc-4ce1-9120-a28640f724cc'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:50.238826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 924/1847] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:49 2022] DELETE /identity/v3/users/e0ebcb3a0f4b48f28f74081da9c36180 => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:50.243226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2e82-57c6-4233-9cb3-c7e5c275956a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:50.243699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2e82-57c6-4233-9cb3-c7e5c275956a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:50.243978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2e82-57c6-4233-9cb3-c7e5c275956a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:50.267527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4bcb2e82-57c6-4233-9cb3-c7e5c275956a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:50.268386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4bcb2e82-57c6-4233-9cb3-c7e5c275956a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:50.338374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 925/1848] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:50.341996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450b6e7-480e-496c-8d02-e574f9ed49d1 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:50.341996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450b6e7-480e-496c-8d02-e574f9ed49d1 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:50.341996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450b6e7-480e-496c-8d02-e574f9ed49d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:50.362640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3450b6e7-480e-496c-8d02-e574f9ed49d1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:50.364132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3450b6e7-480e-496c-8d02-e574f9ed49d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:50.370256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106c6c134f7b418699e3097adb022604', 'request_id': 'req-deb93270-9ebc-4ce1-9120-a28640f724cc'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:50.377376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 926/1849] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:50.378069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450b6e7-480e-496c-8d02-e574f9ed49d1 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:50.379382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3450b6e7-480e-496c-8d02-e574f9ed49d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:50.387703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] The token cache is being invalidate because project 106c6c134f7b418699e3097adb022604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:50.388000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:50.388659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:50.390317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 924/1850] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:49 2022] DELETE /identity/v3/projects/106c6c134f7b418699e3097adb022604 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:50.397395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:50.397395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:50.450332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450b6e7-480e-496c-8d02-e574f9ed49d1 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:50.451309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb93270-9ebc-4ce1-9120-a28640f724cc admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:50.453577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:50.454425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:50.454597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:50.454741 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:50.454852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:50.455253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:50.455528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:50.455796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:50.456832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:50.457696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:50.469719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:50.470137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:50.471228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:50.471953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:50.482459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 925/1851] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:50.483290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:50.484437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 927/1852] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:50.484618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:50.497642 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a53175-86e1-4413-b6c4-579d798e2831 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:50.502431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:50.503072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:50.503357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:50.503621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] PATH_INFO: `/v3/users/a9de993a28d54f5f87d72c90ad4d908a` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:50.505024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9de993a28d54f5f87d72c90ad4d908a)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:50.507062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:50.931118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9de993a28d54f5f87d72c90ad4d908a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:51.115161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9de993a28d54f5f87d72c90ad4d908a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:51.138600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9de993a28d54f5f87d72c90ad4d908a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:51.152972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9de993a28d54f5f87d72c90ad4d908a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:51.154456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9de993a28d54f5f87d72c90ad4d908a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:51.163149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 926/1853] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:50 2022] DELETE /identity/v3/users/a9de993a28d54f5f87d72c90ad4d908a => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:52.241319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:52.241319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:52.241984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:52.241984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:52.254176 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7761c-e414-4d40-8c79-68aadb09085b service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:52.361695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:52.362626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:52.363143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:52.363630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] PATH_INFO: `/v3/projects/1408c73523bb463c88702f63fb3d09de` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:52.365067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1408c73523bb463c88702f63fb3d09de)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:52.367263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:52.399415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e01a7-8fbc-46c9-add4-f65ee12b418e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:52.401630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:52.406082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:52.406799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:52.407181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:52.407689 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] PATH_INFO: `/v3/projects/3bef8cad76964c0f844eea3fdd05eff1` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:52.408763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bef8cad76964c0f844eea3fdd05eff1)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:52.410842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:52.676626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bef8cad76964c0f844eea3fdd05eff1', 'request_id': 'req-946f4ba7-c136-4d9f-83a9-d077be98b7d7'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:52.680082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1408c73523bb463c88702f63fb3d09de', 'request_id': 'req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:52.686998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bef8cad76964c0f844eea3fdd05eff1', 'request_id': 'req-946f4ba7-c136-4d9f-83a9-d077be98b7d7'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:52.700412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1408c73523bb463c88702f63fb3d09de', 'request_id': 'req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:52.907671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bef8cad76964c0f844eea3fdd05eff1', 'request_id': 'req-946f4ba7-c136-4d9f-83a9-d077be98b7d7'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:52.920532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] The token cache is being invalidate because project 3bef8cad76964c0f844eea3fdd05eff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:52.927509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:52.927509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:52.927509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 927/1854] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:52 2022] DELETE /identity/v3/projects/3bef8cad76964c0f844eea3fdd05eff1 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:52.927509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:52.927509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:52.998916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f4ba7-c136-4d9f-83a9-d077be98b7d7 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:53.005264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:53.006004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:53.006284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:53.006553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] PATH_INFO: `/v3/projects/0c652b3ae0534b9489f6374feff11a28` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:53.007987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c652b3ae0534b9489f6374feff11a28)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:53.009920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:53.085280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:53.093111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1408c73523bb463c88702f63fb3d09de', 'request_id': 'req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.100186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] The token cache is being invalidate because project 1408c73523bb463c88702f63fb3d09de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:53.100596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.101335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.103094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 928/1855] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:52 2022] DELETE /identity/v3/projects/1408c73523bb463c88702f63fb3d09de => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:53.106943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:53.108390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:53.198629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a37a3-17a5-4582-ac8f-e8e862e1ba4d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:53.206038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:53.206799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:53.207294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:53.207571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] PATH_INFO: `/v3/users/2b121d3f4bae46ec9e2d50c88a56764d` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:53.209399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b121d3f4bae46ec9e2d50c88a56764d)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:53.211417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:53.229898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c652b3ae0534b9489f6374feff11a28', 'request_id': 'req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.236172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c652b3ae0534b9489f6374feff11a28', 'request_id': 'req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.427352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c652b3ae0534b9489f6374feff11a28', 'request_id': 'req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.439880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] The token cache is being invalidate because project 0c652b3ae0534b9489f6374feff11a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:53.440216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.440953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.443160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 928/1856] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:52 2022] DELETE /identity/v3/projects/0c652b3ae0534b9489f6374feff11a28 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:53.443936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:53.445268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:53.521307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58f7fc-bb83-4317-aca9-d2bfe87dd6e8 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:53.525812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:53.526443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:53.526720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:53.526990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] PATH_INFO: `/v3/users/f380db24d3ad43309f8eb716b7e3c976` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:53.528117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f380db24d3ad43309f8eb716b7e3c976)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:53.530054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:53.618734 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b121d3f4bae46ec9e2d50c88a56764d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.782253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b121d3f4bae46ec9e2d50c88a56764d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.792648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b121d3f4bae46ec9e2d50c88a56764d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.807142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b121d3f4bae46ec9e2d50c88a56764d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.807991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e67e2681-c514-459e-88bc-5f740e3611c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b121d3f4bae46ec9e2d50c88a56764d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.814985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 929/1857] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:53 2022] DELETE /identity/v3/users/2b121d3f4bae46ec9e2d50c88a56764d => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:53.817076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c33e48-1a5a-4501-8603-1591cc330b79 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:53.817076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c33e48-1a5a-4501-8603-1591cc330b79 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:53.817949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c33e48-1a5a-4501-8603-1591cc330b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:53.843115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-59c33e48-1a5a-4501-8603-1591cc330b79 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:53.843115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-59c33e48-1a5a-4501-8603-1591cc330b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:53.938863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 930/1858] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:53.940865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f380db24d3ad43309f8eb716b7e3c976'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:53.953003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed3fab-1ba3-4077-a803-d2a28bc579b0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:53.953378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed3fab-1ba3-4077-a803-d2a28bc579b0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:53.953621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed3fab-1ba3-4077-a803-d2a28bc579b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:53.980226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-72ed3fab-1ba3-4077-a803-d2a28bc579b0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:53.981464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-72ed3fab-1ba3-4077-a803-d2a28bc579b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:53.998763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 931/1859] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:54.006256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed3fab-1ba3-4077-a803-d2a28bc579b0 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:54.016095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed3fab-1ba3-4077-a803-d2a28bc579b0 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:54.024390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb76a9-d7b2-47f9-878d-0911470c495f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:54.024390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb76a9-d7b2-47f9-878d-0911470c495f admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:54.024390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb76a9-d7b2-47f9-878d-0911470c495f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:54.024390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb76a9-d7b2-47f9-878d-0911470c495f admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:54.024812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb76a9-d7b2-47f9-878d-0911470c495f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:54.024956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb76a9-d7b2-47f9-878d-0911470c495f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:54.026742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb76a9-d7b2-47f9-878d-0911470c495f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:54.043581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 932/1860] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:00:54.044472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb76a9-d7b2-47f9-878d-0911470c495f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:54.057035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb76a9-d7b2-47f9-878d-0911470c495f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:54.064161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a860ea9-8c9c-403c-adc5-1c4d4c46f748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:54.064783 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860ea9-8c9c-403c-adc5-1c4d4c46f748 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:54.065067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860ea9-8c9c-403c-adc5-1c4d4c46f748 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:54.065341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860ea9-8c9c-403c-adc5-1c4d4c46f748 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:54.066053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a860ea9-8c9c-403c-adc5-1c4d4c46f748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:54.072394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a860ea9-8c9c-403c-adc5-1c4d4c46f748 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:54.081170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f380db24d3ad43309f8eb716b7e3c976'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:54.118143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f380db24d3ad43309f8eb716b7e3c976'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:54.126329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f380db24d3ad43309f8eb716b7e3c976'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:54.127567 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f380db24d3ad43309f8eb716b7e3c976'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:54.139953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 929/1861] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:53 2022] DELETE /identity/v3/users/f380db24d3ad43309f8eb716b7e3c976 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:54.222096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 933/1862] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:54 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:54.226399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:54.231286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:54.358792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277ab14-5532-43be-a57d-02a5f0b2a374 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:54.439324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e39ea9-c936-426d-bbdd-11456ddb9bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:54.445940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e39ea9-c936-426d-bbdd-11456ddb9bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:54.445940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e39ea9-c936-426d-bbdd-11456ddb9bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:54.445940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e39ea9-c936-426d-bbdd-11456ddb9bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:54.445940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e39ea9-c936-426d-bbdd-11456ddb9bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:54.445940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e39ea9-c936-426d-bbdd-11456ddb9bc2 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:54.542358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 930/1863] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:54 2022] POST /identity/v3/users => generated 288 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:54.546688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a860ea9-8c9c-403c-adc5-1c4d4c46f748 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:54.565469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a860ea9-8c9c-403c-adc5-1c4d4c46f748 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:54.580084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41520b7-4d6e-47a5-a9dd-009a162d8568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:54.581900 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41520b7-4d6e-47a5-a9dd-009a162d8568 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:54.582538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41520b7-4d6e-47a5-a9dd-009a162d8568 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:54.583654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41520b7-4d6e-47a5-a9dd-009a162d8568 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:54.584527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41520b7-4d6e-47a5-a9dd-009a162d8568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:54.587543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41520b7-4d6e-47a5-a9dd-009a162d8568 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:54.604610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 934/1864] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:54.609412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e39ea9-c936-426d-bbdd-11456ddb9bc2 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:54.625886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e39ea9-c936-426d-bbdd-11456ddb9bc2 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:54.633988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf48ca-5f30-415a-a9fe-d097a861aa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:54.634851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf48ca-5f30-415a-a9fe-d097a861aa77 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:54.635306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf48ca-5f30-415a-a9fe-d097a861aa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:54.635728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf48ca-5f30-415a-a9fe-d097a861aa77 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:54.636378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf48ca-5f30-415a-a9fe-d097a861aa77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:54.643236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf48ca-5f30-415a-a9fe-d097a861aa77 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:54.661522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 931/1865] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:54.661742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41520b7-4d6e-47a5-a9dd-009a162d8568 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:54.673750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:54.682422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41520b7-4d6e-47a5-a9dd-009a162d8568 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:54.690897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1b2d9-d86d-4ad5-a224-1b5192241a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:54.691587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1b2d9-d86d-4ad5-a224-1b5192241a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:54.691867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1b2d9-d86d-4ad5-a224-1b5192241a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:54.692165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1b2d9-d86d-4ad5-a224-1b5192241a53 admin admin] PATH_INFO: `/v3/projects/e5d243b1c0624ce58d5a0242766a7de2/users/ca0d48265cfc4ae694449ccb76251c63/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:54.708335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1b2d9-d86d-4ad5-a224-1b5192241a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d243b1c0624ce58d5a0242766a7de2, user_id=ca0d48265cfc4ae694449ccb76251c63, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:54.710915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1b2d9-d86d-4ad5-a224-1b5192241a53 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:54.836650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf48ca-5f30-415a-a9fe-d097a861aa77 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:54.838573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-66cf48ca-5f30-415a-a9fe-d097a861aa77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:54.839308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 935/1866] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:54 2022] PUT /identity/v3/projects/e5d243b1c0624ce58d5a0242766a7de2/users/ca0d48265cfc4ae694449ccb76251c63/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:54.840211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1b2d9-d86d-4ad5-a224-1b5192241a53 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:54.842015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e2b1b2d9-d86d-4ad5-a224-1b5192241a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:54.868715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf48ca-5f30-415a-a9fe-d097a861aa77 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:54.922377 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1b2d9-d86d-4ad5-a224-1b5192241a53 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:54.928265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:54.932044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:54.932044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:54.932044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] PATH_INFO: `/v3/projects/668a1f69935c4da0823d39e382f11ce7` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:54.932044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=668a1f69935c4da0823d39e382f11ce7)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:54.933554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:54.934155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:54.934794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:54.935085 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:54.935350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] PATH_INFO: `/v3/projects/4d5dd415b85d4d1392f43248a7c9b24b` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:54.936546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5dd415b85d4d1392f43248a7c9b24b)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:54.938506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:55.069333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668a1f69935c4da0823d39e382f11ce7', 'request_id': 'req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.079281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668a1f69935c4da0823d39e382f11ce7', 'request_id': 'req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.096242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:55.117715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5dd415b85d4d1392f43248a7c9b24b', 'request_id': 'req-3acf9a51-0f6c-4715-b1aa-77088001ae84'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.124570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5dd415b85d4d1392f43248a7c9b24b', 'request_id': 'req-3acf9a51-0f6c-4715-b1aa-77088001ae84'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.194758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668a1f69935c4da0823d39e382f11ce7', 'request_id': 'req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.208122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] The token cache is being invalidate because project 668a1f69935c4da0823d39e382f11ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:55.208624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.209267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-327ab6e4-9f10-4018-84e5-ae6fbc0d3b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.210945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 936/1867] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:54 2022] DELETE /identity/v3/projects/668a1f69935c4da0823d39e382f11ce7 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:55.212270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fee88-b9a4-47f8-b946-cf1d7168ef3e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.212605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fee88-b9a4-47f8-b946-cf1d7168ef3e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.212960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fee88-b9a4-47f8-b946-cf1d7168ef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.245732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-270fee88-b9a4-47f8-b946-cf1d7168ef3e None None] MFA Rules not processed for user `ca0d48265cfc4ae694449ccb76251c63`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:55.247019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-270fee88-b9a4-47f8-b946-cf1d7168ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:55.268052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5dd415b85d4d1392f43248a7c9b24b', 'request_id': 'req-3acf9a51-0f6c-4715-b1aa-77088001ae84'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.281124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] The token cache is being invalidate because project 4d5dd415b85d4d1392f43248a7c9b24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:55.281429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.282134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3acf9a51-0f6c-4715-b1aa-77088001ae84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:55.289986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 932/1868] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:54 2022] DELETE /identity/v3/projects/4d5dd415b85d4d1392f43248a7c9b24b => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:55.291456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64921c-b88c-4af1-9ad9-b44850a926fc None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.291741 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64921c-b88c-4af1-9ad9-b44850a926fc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.292013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64921c-b88c-4af1-9ad9-b44850a926fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.320980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ac64921c-b88c-4af1-9ad9-b44850a926fc None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:55.324708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ac64921c-b88c-4af1-9ad9-b44850a926fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:55.328815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 937/1869] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:55.330570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979156-22b8-48ef-b1cf-d67b897262c7 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.330857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979156-22b8-48ef-b1cf-d67b897262c7 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.331477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979156-22b8-48ef-b1cf-d67b897262c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.354398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-7b979156-22b8-48ef-b1cf-d67b897262c7 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:55.355614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7b979156-22b8-48ef-b1cf-d67b897262c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:55.404858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 933/1870] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:55.412203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64921c-b88c-4af1-9ad9-b44850a926fc None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:55.414060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ac64921c-b88c-4af1-9ad9-b44850a926fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:55.424131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 938/1871] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:55.428662 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6678cf3-d978-4441-bd49-91fa7f84fd49 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.431702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6678cf3-d978-4441-bd49-91fa7f84fd49 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.434476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6678cf3-d978-4441-bd49-91fa7f84fd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.466271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d6678cf3-d978-4441-bd49-91fa7f84fd49 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:55.468144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d6678cf3-d978-4441-bd49-91fa7f84fd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:55.487986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 939/1872] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:55.491139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da76edc-c82c-41ff-b63a-3385e7f02b00 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.491575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da76edc-c82c-41ff-b63a-3385e7f02b00 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.491999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da76edc-c82c-41ff-b63a-3385e7f02b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.550911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8da76edc-c82c-41ff-b63a-3385e7f02b00 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:55.553658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8da76edc-c82c-41ff-b63a-3385e7f02b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:55.563665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 940/1873] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:55.570135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da76edc-c82c-41ff-b63a-3385e7f02b00 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:55.581946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da76edc-c82c-41ff-b63a-3385e7f02b00 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:55.588575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64921c-b88c-4af1-9ad9-b44850a926fc None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:55.597914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134a015-816a-4988-88be-2d0f5d63378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:55.598127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db49df-4d6f-43d2-83f9-c924ded8e0cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73bfe655c6d34918b21acb8748b9156d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:55.598227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134a015-816a-4988-88be-2d0f5d63378b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.598227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134a015-816a-4988-88be-2d0f5d63378b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.598894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db49df-4d6f-43d2-83f9-c924ded8e0cb service glance] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.598894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db49df-4d6f-43d2-83f9-c924ded8e0cb service glance] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.599033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134a015-816a-4988-88be-2d0f5d63378b admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.599108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db49df-4d6f-43d2-83f9-c924ded8e0cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.599207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134a015-816a-4988-88be-2d0f5d63378b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:55.599207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134a015-816a-4988-88be-2d0f5d63378b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:55.599207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134a015-816a-4988-88be-2d0f5d63378b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:55.610923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db49df-4d6f-43d2-83f9-c924ded8e0cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:55.612640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db49df-4d6f-43d2-83f9-c924ded8e0cb service glance] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:55.621817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 941/1874] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:55.621817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134a015-816a-4988-88be-2d0f5d63378b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:55.631772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134a015-816a-4988-88be-2d0f5d63378b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:55.637212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccca0c0-810d-4a72-b7bf-392e20146f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:55.639683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccca0c0-810d-4a72-b7bf-392e20146f64 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.639683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccca0c0-810d-4a72-b7bf-392e20146f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.639683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccca0c0-810d-4a72-b7bf-392e20146f64 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.639683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccca0c0-810d-4a72-b7bf-392e20146f64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:00:55.639683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccca0c0-810d-4a72-b7bf-392e20146f64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:55.641186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccca0c0-810d-4a72-b7bf-392e20146f64 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:55.642024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 934/1875] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:55.644956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db49df-4d6f-43d2-83f9-c924ded8e0cb service glance] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:55.667692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 942/1876] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:00:55.669870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccca0c0-810d-4a72-b7bf-392e20146f64 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:55.671814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db49df-4d6f-43d2-83f9-c924ded8e0cb service glance] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:55.678032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ef708-a2a1-44b1-b640-282601c0ee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:55.679058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ef708-a2a1-44b1-b640-282601c0ee41 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.679387 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ef708-a2a1-44b1-b640-282601c0ee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.685389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ef708-a2a1-44b1-b640-282601c0ee41 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.685389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ef708-a2a1-44b1-b640-282601c0ee41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:55.685389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ef708-a2a1-44b1-b640-282601c0ee41 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:55.686161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccca0c0-810d-4a72-b7bf-392e20146f64 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:55.695744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093bc35-3320-4d22-976a-b608d4de68cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:55.695744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093bc35-3320-4d22-976a-b608d4de68cc admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.695744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093bc35-3320-4d22-976a-b608d4de68cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.695744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093bc35-3320-4d22-976a-b608d4de68cc admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.696243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093bc35-3320-4d22-976a-b608d4de68cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:55.696243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093bc35-3320-4d22-976a-b608d4de68cc admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:55.812003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 943/1877] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:55 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:55.813184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093bc35-3320-4d22-976a-b608d4de68cc admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:55.815516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4093bc35-3320-4d22-976a-b608d4de68cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:55.837833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 935/1878] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:55 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:55.837833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ef708-a2a1-44b1-b640-282601c0ee41 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:55.904890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093bc35-3320-4d22-976a-b608d4de68cc admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:55.916215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:55.920596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ef708-a2a1-44b1-b640-282601c0ee41 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:55.930027 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9154f-4ec0-4e49-9345-47395d889847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:55.930645 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9154f-4ec0-4e49-9345-47395d889847 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.931086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9154f-4ec0-4e49-9345-47395d889847 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.931521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9154f-4ec0-4e49-9345-47395d889847 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.932157 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9154f-4ec0-4e49-9345-47395d889847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:55.934120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9154f-4ec0-4e49-9345-47395d889847 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:55.981223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:55.981951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:55.982232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:55.982506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] PATH_INFO: `/v3/users/1c8b1ce87d074acd8d608575d3b5c7cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:55.983691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8b1ce87d074acd8d608575d3b5c7cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:55.985730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.042163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 936/1879] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:55 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:56.042936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9154f-4ec0-4e49-9345-47395d889847 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.053643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9154f-4ec0-4e49-9345-47395d889847 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.058625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945abde-9ef1-42cb-a8ed-bbe4b98ac92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.059217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945abde-9ef1-42cb-a8ed-bbe4b98ac92f admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.059497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945abde-9ef1-42cb-a8ed-bbe4b98ac92f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.059772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945abde-9ef1-42cb-a8ed-bbe4b98ac92f admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.060385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945abde-9ef1-42cb-a8ed-bbe4b98ac92f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.062330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945abde-9ef1-42cb-a8ed-bbe4b98ac92f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.227868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 937/1880] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:56 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:56.227868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945abde-9ef1-42cb-a8ed-bbe4b98ac92f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.234540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945abde-9ef1-42cb-a8ed-bbe4b98ac92f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.240030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e9f72-16f0-4cdf-95f3-dd97f84ccfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.240620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e9f72-16f0-4cdf-95f3-dd97f84ccfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.240893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e9f72-16f0-4cdf-95f3-dd97f84ccfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.241159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e9f72-16f0-4cdf-95f3-dd97f84ccfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.241816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e9f72-16f0-4cdf-95f3-dd97f84ccfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.243405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e9f72-16f0-4cdf-95f3-dd97f84ccfa6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.254033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 938/1881] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:56.254871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e9f72-16f0-4cdf-95f3-dd97f84ccfa6 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.274036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e9f72-16f0-4cdf-95f3-dd97f84ccfa6 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.279368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970ef0e-6872-46d2-abc8-c25c1174f8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.279975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970ef0e-6872-46d2-abc8-c25c1174f8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.280251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970ef0e-6872-46d2-abc8-c25c1174f8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.281414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970ef0e-6872-46d2-abc8-c25c1174f8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.281414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970ef0e-6872-46d2-abc8-c25c1174f8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.282791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970ef0e-6872-46d2-abc8-c25c1174f8b0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.290272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8b1ce87d074acd8d608575d3b5c7cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:56.293985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 939/1882] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:56.294851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970ef0e-6872-46d2-abc8-c25c1174f8b0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.304664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970ef0e-6872-46d2-abc8-c25c1174f8b0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.309373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39220d0-c423-483a-938e-796866e39c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.309996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39220d0-c423-483a-938e-796866e39c1c admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.310277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39220d0-c423-483a-938e-796866e39c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.310549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39220d0-c423-483a-938e-796866e39c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.311201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39220d0-c423-483a-938e-796866e39c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.312953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39220d0-c423-483a-938e-796866e39c1c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.324181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 940/1883] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:56.324913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39220d0-c423-483a-938e-796866e39c1c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.335341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39220d0-c423-483a-938e-796866e39c1c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.340163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c4b48-1503-4adb-9ebc-e7af394140ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.340853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c4b48-1503-4adb-9ebc-e7af394140ed admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.341653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c4b48-1503-4adb-9ebc-e7af394140ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.341653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c4b48-1503-4adb-9ebc-e7af394140ed admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.342242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c4b48-1503-4adb-9ebc-e7af394140ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.343912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c4b48-1503-4adb-9ebc-e7af394140ed admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.354623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 941/1884] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:56.355528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c4b48-1503-4adb-9ebc-e7af394140ed admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.365165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c4b48-1503-4adb-9ebc-e7af394140ed admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.369964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d71a3-f548-4234-8050-3f1c998c17a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.370616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d71a3-f548-4234-8050-3f1c998c17a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.370897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d71a3-f548-4234-8050-3f1c998c17a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.371715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d71a3-f548-4234-8050-3f1c998c17a8 admin admin] PATH_INFO: `/v3/projects/2ea3c0d6bec144e08e216604ee25240a/users/ec93488d852c46bb9d813ec09c7ecf28/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.376629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8b1ce87d074acd8d608575d3b5c7cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:56.384288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8b1ce87d074acd8d608575d3b5c7cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:56.397997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8b1ce87d074acd8d608575d3b5c7cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:56.398814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8b1ce87d074acd8d608575d3b5c7cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:56.399389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d71a3-f548-4234-8050-3f1c998c17a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea3c0d6bec144e08e216604ee25240a, user_id=ec93488d852c46bb9d813ec09c7ecf28, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.403515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d71a3-f548-4234-8050-3f1c998c17a8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.405382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 944/1885] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:55 2022] DELETE /identity/v3/users/1c8b1ce87d074acd8d608575d3b5c7cd => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:56.406385 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.408141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:56.476844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4477a-b366-44bc-86d9-f8f91bd9fd08 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.481632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b505dda-37a9-4383-9cbc-6aaa8a329140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.482374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b505dda-37a9-4383-9cbc-6aaa8a329140 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.482740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b505dda-37a9-4383-9cbc-6aaa8a329140 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.487752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b505dda-37a9-4383-9cbc-6aaa8a329140 admin admin] PATH_INFO: `/v3/projects/9b325e9b9d1244d6b81541f14388bdb7/users/a6860f99a3634aa5abffd49229af1578/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.511187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b505dda-37a9-4383-9cbc-6aaa8a329140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b325e9b9d1244d6b81541f14388bdb7, user_id=a6860f99a3634aa5abffd49229af1578, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.514112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b505dda-37a9-4383-9cbc-6aaa8a329140 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.523508 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 942/1886] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:56 2022] PUT /identity/v3/projects/2ea3c0d6bec144e08e216604ee25240a/users/ec93488d852c46bb9d813ec09c7ecf28/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:56.525506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080fa6a-d2d1-4245-8259-9b78b2d77059 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.526003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080fa6a-d2d1-4245-8259-9b78b2d77059 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.526572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080fa6a-d2d1-4245-8259-9b78b2d77059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.546014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0080fa6a-d2d1-4245-8259-9b78b2d77059 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:56.547208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0080fa6a-d2d1-4245-8259-9b78b2d77059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:56.605775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 945/1887] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:56 2022] PUT /identity/v3/projects/9b325e9b9d1244d6b81541f14388bdb7/users/a6860f99a3634aa5abffd49229af1578/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:56.614469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388c2f-cfdb-4322-bc28-07abe1081adc None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.614469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388c2f-cfdb-4322-bc28-07abe1081adc None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.614469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388c2f-cfdb-4322-bc28-07abe1081adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.622746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 943/1888] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:56.624644 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c70808-2005-4b2f-839e-3aad4d9cc785 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.624926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c70808-2005-4b2f-839e-3aad4d9cc785 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.625488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c70808-2005-4b2f-839e-3aad4d9cc785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.637460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:56.660790 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-23c70808-2005-4b2f-839e-3aad4d9cc785 None None] MFA Rules not processed for user `a6860f99a3634aa5abffd49229af1578`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:56.660929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6f388c2f-cfdb-4322-bc28-07abe1081adc None None] MFA Rules not processed for user `ec93488d852c46bb9d813ec09c7ecf28`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:56.662136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-23c70808-2005-4b2f-839e-3aad4d9cc785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:56.663235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6f388c2f-cfdb-4322-bc28-07abe1081adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:56.793436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 946/1889] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:56.794919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f388c2f-cfdb-4322-bc28-07abe1081adc None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.797994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6f388c2f-cfdb-4322-bc28-07abe1081adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:56.811476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 944/1890] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:56.813345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c70808-2005-4b2f-839e-3aad4d9cc785 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.818426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-23c70808-2005-4b2f-839e-3aad4d9cc785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:56.925270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f388c2f-cfdb-4322-bc28-07abe1081adc None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.931816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b278e-4a9a-4c9e-9a02-60fcd02187c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.932853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b278e-4a9a-4c9e-9a02-60fcd02187c6 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.933428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b278e-4a9a-4c9e-9a02-60fcd02187c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.934128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b278e-4a9a-4c9e-9a02-60fcd02187c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.950600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b278e-4a9a-4c9e-9a02-60fcd02187c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.953485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b278e-4a9a-4c9e-9a02-60fcd02187c6 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.967033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c70808-2005-4b2f-839e-3aad4d9cc785 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.967033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a514428-2b08-4ca7-b7a3-80d5fff8926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.967033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a514428-2b08-4ca7-b7a3-80d5fff8926a admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.967033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a514428-2b08-4ca7-b7a3-80d5fff8926a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.967688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a514428-2b08-4ca7-b7a3-80d5fff8926a admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.967688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a514428-2b08-4ca7-b7a3-80d5fff8926a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.967985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 947/1891] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:56.967985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b278e-4a9a-4c9e-9a02-60fcd02187c6 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:56.968253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a514428-2b08-4ca7-b7a3-80d5fff8926a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:56.976744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b278e-4a9a-4c9e-9a02-60fcd02187c6 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:56.982460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2b1cc-336f-46db-8821-5fc3a44de7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:56.983067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2b1cc-336f-46db-8821-5fc3a44de7c0 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:56.983347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2b1cc-336f-46db-8821-5fc3a44de7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:56.983615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2b1cc-336f-46db-8821-5fc3a44de7c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:56.984369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2b1cc-336f-46db-8821-5fc3a44de7c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:56.986498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2b1cc-336f-46db-8821-5fc3a44de7c0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.099960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 948/1893] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:56 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:57.100168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 945/1893] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:56 2022] POST /identity/v3/projects => generated 379 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:57.101302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a514428-2b08-4ca7-b7a3-80d5fff8926a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.102329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2b1cc-336f-46db-8821-5fc3a44de7c0 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.171801 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2b1cc-336f-46db-8821-5fc3a44de7c0 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.175902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a514428-2b08-4ca7-b7a3-80d5fff8926a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.185536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9386-f8be-4171-ae39-743c114c07b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.185536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9386-f8be-4171-ae39-743c114c07b8 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.185536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9386-f8be-4171-ae39-743c114c07b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.185536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9386-f8be-4171-ae39-743c114c07b8 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.185926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23aa43-949e-4091-be10-105cf9da1998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.186020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f9386-f8be-4171-ae39-743c114c07b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.186096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23aa43-949e-4091-be10-105cf9da1998 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.186096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23aa43-949e-4091-be10-105cf9da1998 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.186096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23aa43-949e-4091-be10-105cf9da1998 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.186096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23aa43-949e-4091-be10-105cf9da1998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.186342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f9386-f8be-4171-ae39-743c114c07b8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.186500 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23aa43-949e-4091-be10-105cf9da1998 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.205189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:57.268786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 946/1894] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:57 2022] POST /identity/v3/users => generated 296 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:57.272523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23aa43-949e-4091-be10-105cf9da1998 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.284340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23aa43-949e-4091-be10-105cf9da1998 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.294875 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 949/1895] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:57 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:00:57.295000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebca70a-1730-4a50-a4d6-6ba31b6413fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.295000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebca70a-1730-4a50-a4d6-6ba31b6413fc admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.295169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebca70a-1730-4a50-a4d6-6ba31b6413fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.295361 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebca70a-1730-4a50-a4d6-6ba31b6413fc admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.296058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebca70a-1730-4a50-a4d6-6ba31b6413fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.297543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9386-f8be-4171-ae39-743c114c07b8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.304697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebca70a-1730-4a50-a4d6-6ba31b6413fc admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.311328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9386-f8be-4171-ae39-743c114c07b8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.316779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f791-ec58-49bd-87f4-836931ad7f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.317454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f791-ec58-49bd-87f4-836931ad7f76 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.317815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f791-ec58-49bd-87f4-836931ad7f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.318500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905f791-ec58-49bd-87f4-836931ad7f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.318719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905f791-ec58-49bd-87f4-836931ad7f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.320514 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905f791-ec58-49bd-87f4-836931ad7f76 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.328081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 947/1896] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:57.333431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebca70a-1730-4a50-a4d6-6ba31b6413fc admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.341722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 950/1897] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:00:57.346664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f791-ec58-49bd-87f4-836931ad7f76 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.351078 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebca70a-1730-4a50-a4d6-6ba31b6413fc admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.358113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a96e-dbc2-41b4-b850-3b51d5363c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.359097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a96e-dbc2-41b4-b850-3b51d5363c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.359097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a96e-dbc2-41b4-b850-3b51d5363c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.359437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a96e-dbc2-41b4-b850-3b51d5363c5f admin admin] PATH_INFO: `/v3/projects/a229a6cc30ec4e0abd5af90aab2a339f/users/690030fc753445518feb73f95b3d23cd/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.370701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905f791-ec58-49bd-87f4-836931ad7f76 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.375599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23a1e1-0567-4415-b426-25e6fb6cd847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.376388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23a1e1-0567-4415-b426-25e6fb6cd847 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.376537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23a1e1-0567-4415-b426-25e6fb6cd847 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.376873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23a1e1-0567-4415-b426-25e6fb6cd847 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.377448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23a1e1-0567-4415-b426-25e6fb6cd847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.379078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23a1e1-0567-4415-b426-25e6fb6cd847 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.381021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45a96e-dbc2-41b4-b850-3b51d5363c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a229a6cc30ec4e0abd5af90aab2a339f, user_id=690030fc753445518feb73f95b3d23cd, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.384473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45a96e-dbc2-41b4-b850-3b51d5363c5f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.394252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 951/1898] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:57.395369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23a1e1-0567-4415-b426-25e6fb6cd847 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.414306 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23a1e1-0567-4415-b426-25e6fb6cd847 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.415213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606eac-3ed3-4f60-bdc4-c0cb4c5e858b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.415879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606eac-3ed3-4f60-bdc4-c0cb4c5e858b admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.416155 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606eac-3ed3-4f60-bdc4-c0cb4c5e858b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.416428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606eac-3ed3-4f60-bdc4-c0cb4c5e858b admin admin] PATH_INFO: `/v3/projects/f113306f3a2e4c57805b1976f3500082/users/4067fcf33b454ae6b2f5d0829ac85c0b/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.432596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a606eac-3ed3-4f60-bdc4-c0cb4c5e858b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f113306f3a2e4c57805b1976f3500082, user_id=4067fcf33b454ae6b2f5d0829ac85c0b, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.435832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a606eac-3ed3-4f60-bdc4-c0cb4c5e858b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.486434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 948/1899] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:57 2022] PUT /identity/v3/projects/a229a6cc30ec4e0abd5af90aab2a339f/users/690030fc753445518feb73f95b3d23cd/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:57.489759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a96e-dbc2-41b4-b850-3b51d5363c5f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.492857 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7b45a96e-dbc2-41b4-b850-3b51d5363c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:57.515823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 952/1900] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:57 2022] PUT /identity/v3/projects/f113306f3a2e4c57805b1976f3500082/users/4067fcf33b454ae6b2f5d0829ac85c0b/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:57.515823 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606eac-3ed3-4f60-bdc4-c0cb4c5e858b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.573517 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a96e-dbc2-41b4-b850-3b51d5363c5f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.618449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606eac-3ed3-4f60-bdc4-c0cb4c5e858b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.625028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fb4b7-d6e1-4016-9b27-92105b14fc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.625679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fb4b7-d6e1-4016-9b27-92105b14fc07 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.627255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fb4b7-d6e1-4016-9b27-92105b14fc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.627255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fb4b7-d6e1-4016-9b27-92105b14fc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.627255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fb4b7-d6e1-4016-9b27-92105b14fc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.629329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fb4b7-d6e1-4016-9b27-92105b14fc07 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.640002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 953/1901] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:57.641398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9e6bf-d4a2-47d2-b38f-327e94e42acb None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.641683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9e6bf-d4a2-47d2-b38f-327e94e42acb None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.641976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9e6bf-d4a2-47d2-b38f-327e94e42acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.650757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.651253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.651677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.652160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] PATH_INFO: `/v3/projects/226a05f0ef6546b7a90c6146608089e6` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.653449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=226a05f0ef6546b7a90c6146608089e6)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.655455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.673606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3df9e6bf-d4a2-47d2-b38f-327e94e42acb None None] MFA Rules not processed for user `4067fcf33b454ae6b2f5d0829ac85c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:57.674854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3df9e6bf-d4a2-47d2-b38f-327e94e42acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:57.681999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:57.749681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 954/1902] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:57.750399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9e6bf-d4a2-47d2-b38f-327e94e42acb None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.763570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9e6bf-d4a2-47d2-b38f-327e94e42acb None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.774790 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226a05f0ef6546b7a90c6146608089e6', 'request_id': 'req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:57.784308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226a05f0ef6546b7a90c6146608089e6', 'request_id': 'req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:57.830357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9fb95-1158-4a48-9318-0b924185450d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.831146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9fb95-1158-4a48-9318-0b924185450d admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.831439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9fb95-1158-4a48-9318-0b924185450d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.831702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9fb95-1158-4a48-9318-0b924185450d admin admin] PATH_INFO: `/v3/projects/a229a6cc30ec4e0abd5af90aab2a339f/users/690030fc753445518feb73f95b3d23cd/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.834653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9fb95-1158-4a48-9318-0b924185450d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a229a6cc30ec4e0abd5af90aab2a339f, user_id=690030fc753445518feb73f95b3d23cd, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.837497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9fb95-1158-4a48-9318-0b924185450d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.851555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 955/1903] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:57 2022] PUT /identity/v3/projects/a229a6cc30ec4e0abd5af90aab2a339f/users/690030fc753445518feb73f95b3d23cd/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:57.852426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9fb95-1158-4a48-9318-0b924185450d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:57.924142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:57.925012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226a05f0ef6546b7a90c6146608089e6', 'request_id': 'req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:57.945831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] The token cache is being invalidate because project 226a05f0ef6546b7a90c6146608089e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:57.946123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:57.946906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a9b3f23f-26e1-4d39-adf6-6e253026a1ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:57.950194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 949/1904] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:57 2022] DELETE /identity/v3/projects/226a05f0ef6546b7a90c6146608089e6 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:57.955135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68133af6-6493-4df5-a1c9-e2d30f7bcfad None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.955481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68133af6-6493-4df5-a1c9-e2d30f7bcfad None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.955831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68133af6-6493-4df5-a1c9-e2d30f7bcfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.963246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9fb95-1158-4a48-9318-0b924185450d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:57.968531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee1508-a0e0-4fc0-9547-2933ca2d379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:57.969235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee1508-a0e0-4fc0-9547-2933ca2d379c admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:57.969584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee1508-a0e0-4fc0-9547-2933ca2d379c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:57.970016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee1508-a0e0-4fc0-9547-2933ca2d379c admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:57.970644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee1508-a0e0-4fc0-9547-2933ca2d379c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:57.972629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee1508-a0e0-4fc0-9547-2933ca2d379c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:57.991510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-68133af6-6493-4df5-a1c9-e2d30f7bcfad None None] MFA Rules not processed for user `690030fc753445518feb73f95b3d23cd`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:57.992939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-68133af6-6493-4df5-a1c9-e2d30f7bcfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:58.162201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 956/1905] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:00:57 2022] POST /identity/v3/projects => generated 379 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:58.164000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee1508-a0e0-4fc0-9547-2933ca2d379c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:58.167937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-49ee1508-a0e0-4fc0-9547-2933ca2d379c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:58.237846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 950/1906] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:58.237846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68133af6-6493-4df5-a1c9-e2d30f7bcfad None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:58.237846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-68133af6-6493-4df5-a1c9-e2d30f7bcfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:58.237846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68133af6-6493-4df5-a1c9-e2d30f7bcfad None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:58.322011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee1508-a0e0-4fc0-9547-2933ca2d379c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:58.328231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:58.330496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:58.337935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:58.337935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] PATH_INFO: `/v3/users/569f41455cf948cfb39c90e6ffcc8fca` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:58.337935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=569f41455cf948cfb39c90e6ffcc8fca)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:58.337935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:58.352302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9045c2-1bc5-4dce-af4e-6d37ea724853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:58.353287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9045c2-1bc5-4dce-af4e-6d37ea724853 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:58.353748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9045c2-1bc5-4dce-af4e-6d37ea724853 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:58.354247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9045c2-1bc5-4dce-af4e-6d37ea724853 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:58.355299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9045c2-1bc5-4dce-af4e-6d37ea724853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:58.359611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9045c2-1bc5-4dce-af4e-6d37ea724853 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:58.455365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 951/1907] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:00:58 2022] POST /identity/v3/users => generated 296 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:00:58.459656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9045c2-1bc5-4dce-af4e-6d37ea724853 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:58.463654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4f9045c2-1bc5-4dce-af4e-6d37ea724853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:58.525218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9045c2-1bc5-4dce-af4e-6d37ea724853 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:58.529122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0027e-2c02-4222-ae16-061a348ff7d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:58.529801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0027e-2c02-4222-ae16-061a348ff7d8 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:58.530100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0027e-2c02-4222-ae16-061a348ff7d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:58.530399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0027e-2c02-4222-ae16-061a348ff7d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:58.615961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '569f41455cf948cfb39c90e6ffcc8fca'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.628360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0027e-2c02-4222-ae16-061a348ff7d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:58.631345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0027e-2c02-4222-ae16-061a348ff7d8 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:58.650765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 952/1908] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:58.652360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0027e-2c02-4222-ae16-061a348ff7d8 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:58.672704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0027e-2c02-4222-ae16-061a348ff7d8 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:58.686804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cc17e-eec8-4fba-8270-f6b70810c033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:58.689031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cc17e-eec8-4fba-8270-f6b70810c033 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:58.689031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cc17e-eec8-4fba-8270-f6b70810c033 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:58.689295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018cc17e-eec8-4fba-8270-f6b70810c033 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:58.690553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018cc17e-eec8-4fba-8270-f6b70810c033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:58.693967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018cc17e-eec8-4fba-8270-f6b70810c033 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:58.718366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '569f41455cf948cfb39c90e6ffcc8fca'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.730589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '569f41455cf948cfb39c90e6ffcc8fca'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.731806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 953/1909] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:00:58.738358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cc17e-eec8-4fba-8270-f6b70810c033 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:58.753318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018cc17e-eec8-4fba-8270-f6b70810c033 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:58.757778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '569f41455cf948cfb39c90e6ffcc8fca'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.759839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '569f41455cf948cfb39c90e6ffcc8fca'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:58.759992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2f5af-0856-4ca5-9b1d-222980a4ade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:58.760693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2f5af-0856-4ca5-9b1d-222980a4ade6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:58.760979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2f5af-0856-4ca5-9b1d-222980a4ade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:58.761247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2f5af-0856-4ca5-9b1d-222980a4ade6 admin admin] PATH_INFO: `/v3/projects/22f74ae8f74d457d935f63b6d5026ba1/users/f1eed91fd0f44405b0750d884fbb9051/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:58.780032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 957/1910] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:00:58 2022] DELETE /identity/v3/users/569f41455cf948cfb39c90e6ffcc8fca => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:58.809390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2f5af-0856-4ca5-9b1d-222980a4ade6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f74ae8f74d457d935f63b6d5026ba1, user_id=f1eed91fd0f44405b0750d884fbb9051, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:58.815362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2f5af-0856-4ca5-9b1d-222980a4ade6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:58.912419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:58.913012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 954/1911] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:58 2022] PUT /identity/v3/projects/22f74ae8f74d457d935f63b6d5026ba1/users/f1eed91fd0f44405b0750d884fbb9051/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:58.914105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:59.029139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea75217-2307-41b9-be9c-4959e5b7ac0d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:59.109010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:00:59.113269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7a9ae-bd81-49c4-8c48-1d86a07cb0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:59.114912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7a9ae-bd81-49c4-8c48-1d86a07cb0aa admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.115328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7a9ae-bd81-49c4-8c48-1d86a07cb0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.115704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7a9ae-bd81-49c4-8c48-1d86a07cb0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:59.129984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7a9ae-bd81-49c4-8c48-1d86a07cb0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:59.129984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7a9ae-bd81-49c4-8c48-1d86a07cb0aa admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:59.169127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 958/1912] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:00:59.170634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2f5af-0856-4ca5-9b1d-222980a4ade6 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:59.187043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2f5af-0856-4ca5-9b1d-222980a4ade6 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:59.193669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd38a-34c0-4066-bcbb-716c89d83a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:59.195148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd38a-34c0-4066-bcbb-716c89d83a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.195918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd38a-34c0-4066-bcbb-716c89d83a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.196355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd38a-34c0-4066-bcbb-716c89d83a50 admin admin] PATH_INFO: `/v3/projects/22f74ae8f74d457d935f63b6d5026ba1/users/f1eed91fd0f44405b0750d884fbb9051/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:59.198752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bd38a-34c0-4066-bcbb-716c89d83a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f74ae8f74d457d935f63b6d5026ba1, user_id=f1eed91fd0f44405b0750d884fbb9051, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:59.201756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bd38a-34c0-4066-bcbb-716c89d83a50 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:59.222235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 955/1913] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:00:59 2022] PUT /identity/v3/projects/22f74ae8f74d457d935f63b6d5026ba1/users/f1eed91fd0f44405b0750d884fbb9051/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:00:59.232922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435ae93-fa98-46af-9078-0578b33fab07 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.233600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435ae93-fa98-46af-9078-0578b33fab07 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.234222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435ae93-fa98-46af-9078-0578b33fab07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:59.279862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-9435ae93-fa98-46af-9078-0578b33fab07 None None] MFA Rules not processed for user `f1eed91fd0f44405b0750d884fbb9051`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:59.281983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9435ae93-fa98-46af-9078-0578b33fab07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:59.420080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 959/1914] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:00:59.441170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd38a-34c0-4066-bcbb-716c89d83a50 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:59.444201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-719bd38a-34c0-4066-bcbb-716c89d83a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:59.500352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435ae93-fa98-46af-9078-0578b33fab07 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:59.502872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9435ae93-fa98-46af-9078-0578b33fab07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:59.588194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd38a-34c0-4066-bcbb-716c89d83a50 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:59.602664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a83e4-339f-4a09-918b-bba8be109ed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:59.603323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a83e4-339f-4a09-918b-bba8be109ed2 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.603922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a83e4-339f-4a09-918b-bba8be109ed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.605154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a83e4-339f-4a09-918b-bba8be109ed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:59.638682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a83e4-339f-4a09-918b-bba8be109ed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:59.641650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a83e4-339f-4a09-918b-bba8be109ed2 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:59.671217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 956/1915] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:59.671985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435ae93-fa98-46af-9078-0578b33fab07 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:59.674496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd7a0c-521d-4ec7-837c-06086b9c84c4 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.675204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd7a0c-521d-4ec7-837c-06086b9c84c4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.675641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd7a0c-521d-4ec7-837c-06086b9c84c4 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:59.677885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:59.681194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 957/1916] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 17:00:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:00:59.681493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.681493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.681493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] PATH_INFO: `/v3/projects/080efe4258ee404cb87fb83ec6a6aa95` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:59.681493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=080efe4258ee404cb87fb83ec6a6aa95)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:59.682737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:59.686214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f112e4-f29b-4fdb-8b4d-78ce4c8e450d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.686564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f112e4-f29b-4fdb-8b4d-78ce4c8e450d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.687067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f112e4-f29b-4fdb-8b4d-78ce4c8e450d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:59.711502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-18f112e4-f29b-4fdb-8b4d-78ce4c8e450d None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:00:59.717584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-18f112e4-f29b-4fdb-8b4d-78ce4c8e450d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:59.811399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 958/1917] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 17:00:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:00:59.812423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f112e4-f29b-4fdb-8b4d-78ce4c8e450d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:59.822397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f112e4-f29b-4fdb-8b4d-78ce4c8e450d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:59.826247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:00:59.827084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.827497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.827677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:00:59.829824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:59.873746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '080efe4258ee404cb87fb83ec6a6aa95', 'request_id': 'req-31863f96-b946-4d31-b54d-6eb50e4ca859'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.880965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '080efe4258ee404cb87fb83ec6a6aa95', 'request_id': 'req-31863f96-b946-4d31-b54d-6eb50e4ca859'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.898411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:00:59.899958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:00:59.909075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 959/1918] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:00:59.909933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:00:59.911913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:00:59.928136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6061ad1-68de-4904-8632-702b26855198 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:00:59.970510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '080efe4258ee404cb87fb83ec6a6aa95', 'request_id': 'req-31863f96-b946-4d31-b54d-6eb50e4ca859'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.987352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] The token cache is being invalidate because project 080efe4258ee404cb87fb83ec6a6aa95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:00:59.988034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.989223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-31863f96-b946-4d31-b54d-6eb50e4ca859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:00:59.991017 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 960/1919] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:00:59 2022] DELETE /identity/v3/projects/080efe4258ee404cb87fb83ec6a6aa95 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:00:59.999704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854ab70-ad39-43cc-93e8-5d4b392b383a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:00:59.999704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854ab70-ad39-43cc-93e8-5d4b392b383a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:00:59.999704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854ab70-ad39-43cc-93e8-5d4b392b383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.034742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f854ab70-ad39-43cc-93e8-5d4b392b383a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:00.037319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.038239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.038727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.039253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.041943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:00.042415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f854ab70-ad39-43cc-93e8-5d4b392b383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:00.060641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.060641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.080060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 960/1920] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:00:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:00.093047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.098780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:00.176288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:00.214087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 961/1921] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:00.223824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c733ebf-78c1-4935-ae70-975ee5e19e4b None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.224184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c733ebf-78c1-4935-ae70-975ee5e19e4b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.224529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c733ebf-78c1-4935-ae70-975ee5e19e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.231532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f85272-6ac4-4882-a575-634f4d7aba92 service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.236667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f3d1f-fec9-49b2-ba8f-0b6d4051cdd9 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690030fc753445518feb73f95b3d23cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a229a6cc30ec4e0abd5af90aab2a339f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.237325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f3d1f-fec9-49b2-ba8f-0b6d4051cdd9 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.237613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f3d1f-fec9-49b2-ba8f-0b6d4051cdd9 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.237934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f3d1f-fec9-49b2-ba8f-0b6d4051cdd9 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.238574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f3d1f-fec9-49b2-ba8f-0b6d4051cdd9 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:00.238903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f3d1f-fec9-49b2-ba8f-0b6d4051cdd9 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.240762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f3d1f-fec9-49b2-ba8f-0b6d4051cdd9 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.259833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-7c733ebf-78c1-4935-ae70-975ee5e19e4b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:00.261065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7c733ebf-78c1-4935-ae70-975ee5e19e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:00.265367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 961/1922] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:00.272645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 962/1923] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:00.279680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f3d1f-fec9-49b2-ba8f-0b6d4051cdd9 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.282659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c733ebf-78c1-4935-ae70-975ee5e19e4b None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.293607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c733ebf-78c1-4935-ae70-975ee5e19e4b None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.302656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f3d1f-fec9-49b2-ba8f-0b6d4051cdd9 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.304779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22087bbc-64c1-4b0a-9ec1-1400de1aea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.305617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22087bbc-64c1-4b0a-9ec1-1400de1aea4f admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.306677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22087bbc-64c1-4b0a-9ec1-1400de1aea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.307631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22087bbc-64c1-4b0a-9ec1-1400de1aea4f admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.308446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22087bbc-64c1-4b0a-9ec1-1400de1aea4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:00.308754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3098a5-b238-4a60-baad-b0179cb239b4 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690030fc753445518feb73f95b3d23cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a229a6cc30ec4e0abd5af90aab2a339f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.309102 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22087bbc-64c1-4b0a-9ec1-1400de1aea4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.309490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3098a5-b238-4a60-baad-b0179cb239b4 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.309898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3098a5-b238-4a60-baad-b0179cb239b4 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.310278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3098a5-b238-4a60-baad-b0179cb239b4 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.310963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3098a5-b238-4a60-baad-b0179cb239b4 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.311640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22087bbc-64c1-4b0a-9ec1-1400de1aea4f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.313076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3098a5-b238-4a60-baad-b0179cb239b4 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.326890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 963/1924] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:00.332175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22087bbc-64c1-4b0a-9ec1-1400de1aea4f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.343124 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22087bbc-64c1-4b0a-9ec1-1400de1aea4f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.348113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c7e2f-0beb-4885-9b9c-34ee54d40e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.348738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c7e2f-0beb-4885-9b9c-34ee54d40e9c admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.349019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c7e2f-0beb-4885-9b9c-34ee54d40e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.349291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c7e2f-0beb-4885-9b9c-34ee54d40e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.349977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c7e2f-0beb-4885-9b9c-34ee54d40e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.354471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c7e2f-0beb-4885-9b9c-34ee54d40e9c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.400994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 962/1925] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:00 2022] POST /identity/v3/projects => generated 355 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:00.404986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3098a5-b238-4a60-baad-b0179cb239b4 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.472771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3098a5-b238-4a60-baad-b0179cb239b4 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.475713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 964/1926] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:00 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:00.479024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b009fc6-3da3-4dcc-9c40-f0946e83c20a tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690030fc753445518feb73f95b3d23cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a229a6cc30ec4e0abd5af90aab2a339f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.479654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b009fc6-3da3-4dcc-9c40-f0946e83c20a tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.479938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b009fc6-3da3-4dcc-9c40-f0946e83c20a tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.480217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b009fc6-3da3-4dcc-9c40-f0946e83c20a tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.481042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b009fc6-3da3-4dcc-9c40-f0946e83c20a tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:00.481352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b009fc6-3da3-4dcc-9c40-f0946e83c20a tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.482849 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c7e2f-0beb-4885-9b9c-34ee54d40e9c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.486931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b009fc6-3da3-4dcc-9c40-f0946e83c20a tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.501056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 963/1927] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:01:00.572772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c7e2f-0beb-4885-9b9c-34ee54d40e9c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.588758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae25e1c-4e41-4789-a592-2488a1f710fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.588758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae25e1c-4e41-4789-a592-2488a1f710fe admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.588758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae25e1c-4e41-4789-a592-2488a1f710fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.588758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae25e1c-4e41-4789-a592-2488a1f710fe admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.589186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae25e1c-4e41-4789-a592-2488a1f710fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.589186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae25e1c-4e41-4789-a592-2488a1f710fe admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.627608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b009fc6-3da3-4dcc-9c40-f0946e83c20a tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.648944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b009fc6-3da3-4dcc-9c40-f0946e83c20a tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.719475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:00.728023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 965/1928] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:00 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:00.732485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae25e1c-4e41-4789-a592-2488a1f710fe admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.753642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae25e1c-4e41-4789-a592-2488a1f710fe admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.759479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690030fc753445518feb73f95b3d23cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a229a6cc30ec4e0abd5af90aab2a339f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.760121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.760664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.761252 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] PATH_INFO: `/v3/projects/01c1a2b0be4049edac01e044f35d4239` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.763675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Authorizing `identity:delete_project(project_id=01c1a2b0be4049edac01e044f35d4239)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.767425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c1513-a645-43f7-9fe4-cacf2acff1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.768097 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c1513-a645-43f7-9fe4-cacf2acff1ea admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.768387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c1513-a645-43f7-9fe4-cacf2acff1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.768659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c1513-a645-43f7-9fe4-cacf2acff1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.769013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.769413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c1513-a645-43f7-9fe4-cacf2acff1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.771937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c1513-a645-43f7-9fe4-cacf2acff1ea admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.788803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 966/1929] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:00.791897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c1513-a645-43f7-9fe4-cacf2acff1ea admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.803835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c1513-a645-43f7-9fe4-cacf2acff1ea admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.809741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813512-11b2-4365-a674-96a63b8c6c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.810602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813512-11b2-4365-a674-96a63b8c6c05 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.811011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813512-11b2-4365-a674-96a63b8c6c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.811248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813512-11b2-4365-a674-96a63b8c6c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.812172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8813512-11b2-4365-a674-96a63b8c6c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.813909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8813512-11b2-4365-a674-96a63b8c6c05 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:00.827366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 967/1930] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:00.835454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813512-11b2-4365-a674-96a63b8c6c05 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:00.847784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813512-11b2-4365-a674-96a63b8c6c05 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:00.854345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bec5d3-56ce-4eca-80d3-0c0fc3e99972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:00.855626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bec5d3-56ce-4eca-80d3-0c0fc3e99972 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:00.856556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bec5d3-56ce-4eca-80d3-0c0fc3e99972 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:00.856845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bec5d3-56ce-4eca-80d3-0c0fc3e99972 admin admin] PATH_INFO: `/v3/projects/9bfaaf7add874364bb2441f8da5fca4c/users/1ee4b3093c594163a4f19e6bb4f4886c/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:00.879323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bec5d3-56ce-4eca-80d3-0c0fc3e99972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bfaaf7add874364bb2441f8da5fca4c, user_id=1ee4b3093c594163a4f19e6bb4f4886c, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:00.882091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bec5d3-56ce-4eca-80d3-0c0fc3e99972 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:01.061274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:01.066761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 968/1931] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:00 2022] PUT /identity/v3/projects/9bfaaf7add874364bb2441f8da5fca4c/users/1ee4b3093c594163a4f19e6bb4f4886c/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:01.074161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c1a2b0be4049edac01e044f35d4239', 'request_id': 'req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.080747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c1a2b0be4049edac01e044f35d4239', 'request_id': 'req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.082881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d922-575e-4dbf-89ff-8fcfe92046e2 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:01.093714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d922-575e-4dbf-89ff-8fcfe92046e2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:01.093714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d922-575e-4dbf-89ff-8fcfe92046e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:01.115169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1767d922-575e-4dbf-89ff-8fcfe92046e2 None None] MFA Rules not processed for user `1ee4b3093c594163a4f19e6bb4f4886c`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:01.116376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1767d922-575e-4dbf-89ff-8fcfe92046e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:01.191579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 969/1932] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:01.196459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767d922-575e-4dbf-89ff-8fcfe92046e2 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:01.284153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767d922-575e-4dbf-89ff-8fcfe92046e2 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:01.290092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5710f-b9b2-4c11-8d02-4b2bca72cf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:01.290727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5710f-b9b2-4c11-8d02-4b2bca72cf83 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:01.290995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5710f-b9b2-4c11-8d02-4b2bca72cf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:01.291270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5710f-b9b2-4c11-8d02-4b2bca72cf83 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:01.291926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5710f-b9b2-4c11-8d02-4b2bca72cf83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:01.294298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5710f-b9b2-4c11-8d02-4b2bca72cf83 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:01.318207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c1a2b0be4049edac01e044f35d4239', 'request_id': 'req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.342950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] The token cache is being invalidate because project 01c1a2b0be4049edac01e044f35d4239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:01.343385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.347052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:01.349830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 964/1933] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:00 2022] DELETE /identity/v3/projects/01c1a2b0be4049edac01e044f35d4239 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:01.427799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 970/1934] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:01 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:01.432748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:01.434363 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:01.461959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d427f-e517-4439-b04e-1ff7d1ad43e0 tempest-VolumeQuotaClassesTest-527890234 tempest-VolumeQuotaClassesTest-527890234-project-admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:01.585025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc88ad-75d7-46a4-b1ce-811f71039d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:01.585372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc88ad-75d7-46a4-b1ce-811f71039d45 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:01.585654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc88ad-75d7-46a4-b1ce-811f71039d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:01.586186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc88ad-75d7-46a4-b1ce-811f71039d45 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:01.587678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc88ad-75d7-46a4-b1ce-811f71039d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:01.595052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc88ad-75d7-46a4-b1ce-811f71039d45 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:01.720459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 965/1935] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:01 2022] POST /identity/v3/users => generated 300 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:01.724657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5710f-b9b2-4c11-8d02-4b2bca72cf83 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:01.740969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5710f-b9b2-4c11-8d02-4b2bca72cf83 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:01.746213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c66cf-415e-42af-9322-e1a7fb87df8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:01.746828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c66cf-415e-42af-9322-e1a7fb87df8f admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:01.747129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c66cf-415e-42af-9322-e1a7fb87df8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:01.747454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c66cf-415e-42af-9322-e1a7fb87df8f admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:01.748293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c66cf-415e-42af-9322-e1a7fb87df8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:01.750028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c66cf-415e-42af-9322-e1a7fb87df8f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:01.763319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 971/1936] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:01.770194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc88ad-75d7-46a4-b1ce-811f71039d45 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:01.782331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc88ad-75d7-46a4-b1ce-811f71039d45 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:01.787726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afed4e-bede-4b77-b93c-cfa548781b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:01.788394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afed4e-bede-4b77-b93c-cfa548781b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:01.788671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afed4e-bede-4b77-b93c-cfa548781b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:01.788937 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afed4e-bede-4b77-b93c-cfa548781b14 admin admin] PATH_INFO: `/v3/projects/e18bf8642fce44b1997d0aac1c109cb3/users/93409e04809c46c49a9f4b8b558760ef/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:01.806088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afed4e-bede-4b77-b93c-cfa548781b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18bf8642fce44b1997d0aac1c109cb3, user_id=93409e04809c46c49a9f4b8b558760ef, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:01.808727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afed4e-bede-4b77-b93c-cfa548781b14 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:01.929759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 966/1937] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:01 2022] PUT /identity/v3/projects/e18bf8642fce44b1997d0aac1c109cb3/users/93409e04809c46c49a9f4b8b558760ef/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:01.934594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c66cf-415e-42af-9322-e1a7fb87df8f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:01.951082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c66cf-415e-42af-9322-e1a7fb87df8f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:02.071606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d0400-7303-4f62-b230-ca3b6ebd3fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:02.073384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d0400-7303-4f62-b230-ca3b6ebd3fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.074865 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d0400-7303-4f62-b230-ca3b6ebd3fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.075740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d0400-7303-4f62-b230-ca3b6ebd3fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.076681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d0400-7303-4f62-b230-ca3b6ebd3fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:02.078748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d0400-7303-4f62-b230-ca3b6ebd3fc3 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:02.099551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 972/1938] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:02.103570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afed4e-bede-4b77-b93c-cfa548781b14 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:02.123283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afed4e-bede-4b77-b93c-cfa548781b14 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:02.131347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200c70b-0e80-4b68-8dc2-5b0248e0932a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:02.132041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c70b-0e80-4b68-8dc2-5b0248e0932a admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.132465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c70b-0e80-4b68-8dc2-5b0248e0932a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.132934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c70b-0e80-4b68-8dc2-5b0248e0932a admin admin] PATH_INFO: `/v3/projects/e18bf8642fce44b1997d0aac1c109cb3/users/93409e04809c46c49a9f4b8b558760ef/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.136787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c70b-0e80-4b68-8dc2-5b0248e0932a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18bf8642fce44b1997d0aac1c109cb3, user_id=93409e04809c46c49a9f4b8b558760ef, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:02.140182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c70b-0e80-4b68-8dc2-5b0248e0932a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:02.165017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 967/1939] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:02 2022] PUT /identity/v3/projects/e18bf8642fce44b1997d0aac1c109cb3/users/93409e04809c46c49a9f4b8b558760ef/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:02.177638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c03af8-0465-40bb-919b-e9b92f6e0b72 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.177984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c03af8-0465-40bb-919b-e9b92f6e0b72 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.178283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c03af8-0465-40bb-919b-e9b92f6e0b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.239983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-43c03af8-0465-40bb-919b-e9b92f6e0b72 None None] MFA Rules not processed for user `93409e04809c46c49a9f4b8b558760ef`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:02.241298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-43c03af8-0465-40bb-919b-e9b92f6e0b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:02.263716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200c70b-0e80-4b68-8dc2-5b0248e0932a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:02.268669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4200c70b-0e80-4b68-8dc2-5b0248e0932a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:02.376366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200c70b-0e80-4b68-8dc2-5b0248e0932a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:02.381522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:02.382371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.382650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.382921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] PATH_INFO: `/v3/users/a6860f99a3634aa5abffd49229af1578` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.384360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6860f99a3634aa5abffd49229af1578)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:02.386537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:02.433043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 973/1940] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:02.447584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c03af8-0465-40bb-919b-e9b92f6e0b72 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:02.453581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-43c03af8-0465-40bb-919b-e9b92f6e0b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:02.526370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c03af8-0465-40bb-919b-e9b92f6e0b72 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:02.535133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955297b-36ed-4523-8171-08325ebbb403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:02.536447 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955297b-36ed-4523-8171-08325ebbb403 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.536679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955297b-36ed-4523-8171-08325ebbb403 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.536947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955297b-36ed-4523-8171-08325ebbb403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.548336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955297b-36ed-4523-8171-08325ebbb403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:02.550840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955297b-36ed-4523-8171-08325ebbb403 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:02.560874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 974/1941] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:02.589359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955297b-36ed-4523-8171-08325ebbb403 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:02.607882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955297b-36ed-4523-8171-08325ebbb403 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:02.614435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:02.615103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.615103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.615282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] PATH_INFO: `/v3/users/1ee4b3093c594163a4f19e6bb4f4886c` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.617262 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee4b3093c594163a4f19e6bb4f4886c)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:02.619369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:02.686468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6860f99a3634aa5abffd49229af1578'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.804192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6860f99a3634aa5abffd49229af1578'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.817853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6860f99a3634aa5abffd49229af1578'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.833900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6860f99a3634aa5abffd49229af1578'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.834718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a225c7b6-cdd5-4938-8688-f2676531a810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6860f99a3634aa5abffd49229af1578'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.845113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 968/1942] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:02 2022] DELETE /identity/v3/users/a6860f99a3634aa5abffd49229af1578 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:02.850067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725be13-9d5b-48d4-9868-38c5900e5c53 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.850354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725be13-9d5b-48d4-9868-38c5900e5c53 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.850622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725be13-9d5b-48d4-9868-38c5900e5c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.869028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f725be13-9d5b-48d4-9868-38c5900e5c53 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:02.869575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee4b3093c594163a4f19e6bb4f4886c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.870183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f725be13-9d5b-48d4-9868-38c5900e5c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:02.929444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee4b3093c594163a4f19e6bb4f4886c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.937555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee4b3093c594163a4f19e6bb4f4886c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.938702 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 969/1943] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:02.951095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725be13-9d5b-48d4-9868-38c5900e5c53 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:02.952665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f725be13-9d5b-48d4-9868-38c5900e5c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:02.956652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee4b3093c594163a4f19e6bb4f4886c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.958131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e4fcdb67-ace4-430e-8c22-1eb06da4ce74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee4b3093c594163a4f19e6bb4f4886c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:02.965250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 975/1944] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:02 2022] DELETE /identity/v3/users/1ee4b3093c594163a4f19e6bb4f4886c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:02.969906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d2953-2f78-4077-a1dc-ae75bcffdad6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:02.970183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d2953-2f78-4077-a1dc-ae75bcffdad6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:02.970467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d2953-2f78-4077-a1dc-ae75bcffdad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:02.989935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4f8d2953-2f78-4077-a1dc-ae75bcffdad6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:02.991161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4f8d2953-2f78-4077-a1dc-ae75bcffdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:03.027406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725be13-9d5b-48d4-9868-38c5900e5c53 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.057245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 976/1945] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:03.067887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d2953-2f78-4077-a1dc-ae75bcffdad6 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.069297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4f8d2953-2f78-4077-a1dc-ae75bcffdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:03.090865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.091495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.091778 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.092055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.093808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:03.106918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.108360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.118319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 970/1946] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:03.127538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d2953-2f78-4077-a1dc-ae75bcffdad6 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.131675 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cbf3e-7be5-4f35-a3c7-af4128631f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.132284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cbf3e-7be5-4f35-a3c7-af4128631f62 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.132575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cbf3e-7be5-4f35-a3c7-af4128631f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.132843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cbf3e-7be5-4f35-a3c7-af4128631f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.136180 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:03.145416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cbf3e-7be5-4f35-a3c7-af4128631f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.146986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cbf3e-7be5-4f35-a3c7-af4128631f62 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.158450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 977/1947] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:03.297521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:03.766544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.767984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:03.790154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cbf3e-7be5-4f35-a3c7-af4128631f62 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:03.791831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-da0cbf3e-7be5-4f35-a3c7-af4128631f62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:03.806377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e16e9e-ffa7-4562-a519-a6b0556a12a4 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.814569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cbf3e-7be5-4f35-a3c7-af4128631f62 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:03.822134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.830187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.830187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.830187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] PATH_INFO: `/v3/projects/9b325e9b9d1244d6b81541f14388bdb7` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.830187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b325e9b9d1244d6b81541f14388bdb7)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.830187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.831055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:03.831055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:03.831055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:03.831055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] PATH_INFO: `/v3/projects/9bfaaf7add874364bb2441f8da5fca4c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:03.833251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bfaaf7add874364bb2441f8da5fca4c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:03.835394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:03.985243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bfaaf7add874364bb2441f8da5fca4c', 'request_id': 'req-94736933-f6a5-49df-82c4-54f6e60f0b30'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:03.987293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b325e9b9d1244d6b81541f14388bdb7', 'request_id': 'req-b04d8bd2-f619-47bf-b405-83913b7a7252'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:03.998096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b325e9b9d1244d6b81541f14388bdb7', 'request_id': 'req-b04d8bd2-f619-47bf-b405-83913b7a7252'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.003076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bfaaf7add874364bb2441f8da5fca4c', 'request_id': 'req-94736933-f6a5-49df-82c4-54f6e60f0b30'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.077251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b325e9b9d1244d6b81541f14388bdb7', 'request_id': 'req-b04d8bd2-f619-47bf-b405-83913b7a7252'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.089273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] The token cache is being invalidate because project 9b325e9b9d1244d6b81541f14388bdb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:04.089563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.090267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.091963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 978/1948] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:03 2022] DELETE /identity/v3/projects/9b325e9b9d1244d6b81541f14388bdb7 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:04.103785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.105305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:04.179694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d8bd2-f619-47bf-b405-83913b7a7252 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:04.184811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.185579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.185887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.186188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] PATH_INFO: `/v3/users/690030fc753445518feb73f95b3d23cd` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.187321 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=690030fc753445518feb73f95b3d23cd)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.189225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.236931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bfaaf7add874364bb2441f8da5fca4c', 'request_id': 'req-94736933-f6a5-49df-82c4-54f6e60f0b30'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.253990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] The token cache is being invalidate because project 9bfaaf7add874364bb2441f8da5fca4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:04.254285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.254940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.256429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 971/1949] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:03 2022] DELETE /identity/v3/projects/9bfaaf7add874364bb2441f8da5fca4c => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:04.260343 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:04.261688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:04.272532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:04.335449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94736933-f6a5-49df-82c4-54f6e60f0b30 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:04.344269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:04.344914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:04.345216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:04.345484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] PATH_INFO: `/v3/users/93409e04809c46c49a9f4b8b558760ef` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:04.346832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93409e04809c46c49a9f4b8b558760ef)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:04.350090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:04.380526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:04.469616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '690030fc753445518feb73f95b3d23cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.592772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '690030fc753445518feb73f95b3d23cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.602088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '690030fc753445518feb73f95b3d23cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.614844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '690030fc753445518feb73f95b3d23cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.615569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '690030fc753445518feb73f95b3d23cd'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.622291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 979/1950] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:04 2022] DELETE /identity/v3/users/690030fc753445518feb73f95b3d23cd => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:04.635798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93409e04809c46c49a9f4b8b558760ef'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.775104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93409e04809c46c49a9f4b8b558760ef'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.791683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93409e04809c46c49a9f4b8b558760ef'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.802521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93409e04809c46c49a9f4b8b558760ef'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.803803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93409e04809c46c49a9f4b8b558760ef'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:04.815893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 972/1951] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:04 2022] DELETE /identity/v3/users/93409e04809c46c49a9f4b8b558760ef => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:05.209310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.209310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:05.241038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:05.284884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb8adc-0af4-4ad8-a6af-b62020fc5079 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.341665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.342628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.345088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:05.345584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.345584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.345584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] PATH_INFO: `/v3/projects/a229a6cc30ec4e0abd5af90aab2a339f` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.345584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a229a6cc30ec4e0abd5af90aab2a339f)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.347609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.359017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4c0d1-4f33-415c-a2e5-17aac7489920 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.366706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:05.369111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.369111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.369111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.369404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] PATH_INFO: `/v3/projects/e18bf8642fce44b1997d0aac1c109cb3` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.370552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18bf8642fce44b1997d0aac1c109cb3)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.372998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.468418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a229a6cc30ec4e0abd5af90aab2a339f', 'request_id': 'req-a9e9534a-168d-4edb-9fe9-c657ea55d265'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.475317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a229a6cc30ec4e0abd5af90aab2a339f', 'request_id': 'req-a9e9534a-168d-4edb-9fe9-c657ea55d265'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.496902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18bf8642fce44b1997d0aac1c109cb3', 'request_id': 'req-e41665d6-24a3-45bd-8060-d6e58f099f13'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.503133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18bf8642fce44b1997d0aac1c109cb3', 'request_id': 'req-e41665d6-24a3-45bd-8060-d6e58f099f13'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.599008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a229a6cc30ec4e0abd5af90aab2a339f', 'request_id': 'req-a9e9534a-168d-4edb-9fe9-c657ea55d265'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.610500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] The token cache is being invalidate because project a229a6cc30ec4e0abd5af90aab2a339f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:05.610816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.611456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a9e9534a-168d-4edb-9fe9-c657ea55d265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.613025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 980/1952] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:05 2022] DELETE /identity/v3/projects/a229a6cc30ec4e0abd5af90aab2a339f => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:05.619217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb813e4-ddaf-4cdc-bb4c-002a5de59c71 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.619543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb813e4-ddaf-4cdc-bb4c-002a5de59c71 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.619857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb813e4-ddaf-4cdc-bb4c-002a5de59c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.646863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1eb813e4-ddaf-4cdc-bb4c-002a5de59c71 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:05.648068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1eb813e4-ddaf-4cdc-bb4c-002a5de59c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:05.666016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18bf8642fce44b1997d0aac1c109cb3', 'request_id': 'req-e41665d6-24a3-45bd-8060-d6e58f099f13'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.677933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] The token cache is being invalidate because project e18bf8642fce44b1997d0aac1c109cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:05.678223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.678967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e41665d6-24a3-45bd-8060-d6e58f099f13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:05.680509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 973/1953] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:05 2022] DELETE /identity/v3/projects/e18bf8642fce44b1997d0aac1c109cb3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:05.690206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08576-dfa2-40eb-82ac-be9be425ebab None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.690490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08576-dfa2-40eb-82ac-be9be425ebab None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.690760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08576-dfa2-40eb-82ac-be9be425ebab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.721614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-41d08576-dfa2-40eb-82ac-be9be425ebab None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:05.721614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-41d08576-dfa2-40eb-82ac-be9be425ebab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:05.721762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 981/1954] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:05.727019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85640-0b3c-4695-af7a-c8425e7e896c None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.729900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 974/1955] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:05.730172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85640-0b3c-4695-af7a-c8425e7e896c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.730450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85640-0b3c-4695-af7a-c8425e7e896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.734676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ad83f-3136-4779-b592-e99f33df1ff3 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.734966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ad83f-3136-4779-b592-e99f33df1ff3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.735243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ad83f-3136-4779-b592-e99f33df1ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.750231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-39a85640-0b3c-4695-af7a-c8425e7e896c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:05.757114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-39a85640-0b3c-4695-af7a-c8425e7e896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:05.757253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-8d1ad83f-3136-4779-b592-e99f33df1ff3 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:05.757253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8d1ad83f-3136-4779-b592-e99f33df1ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:05.761933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 982/1956] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:05.765159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85640-0b3c-4695-af7a-c8425e7e896c None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.765908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 975/1957] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:05.770627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ad83f-3136-4779-b592-e99f33df1ff3 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.775193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85640-0b3c-4695-af7a-c8425e7e896c None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.780440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ad83f-3136-4779-b592-e99f33df1ff3 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.782772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60671480-58a9-4fb7-9f0b-6524b72b7b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.783449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60671480-58a9-4fb7-9f0b-6524b72b7b27 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.783724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60671480-58a9-4fb7-9f0b-6524b72b7b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.783994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60671480-58a9-4fb7-9f0b-6524b72b7b27 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.784637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60671480-58a9-4fb7-9f0b-6524b72b7b27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:05.784938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60671480-58a9-4fb7-9f0b-6524b72b7b27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.787114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a5d9f-c064-4987-bfe0-1ba1cc0b2454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.788409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60671480-58a9-4fb7-9f0b-6524b72b7b27 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.788512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a5d9f-c064-4987-bfe0-1ba1cc0b2454 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.788512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a5d9f-c064-4987-bfe0-1ba1cc0b2454 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.788512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a5d9f-c064-4987-bfe0-1ba1cc0b2454 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.789088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a5d9f-c064-4987-bfe0-1ba1cc0b2454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:05.790112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a5d9f-c064-4987-bfe0-1ba1cc0b2454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.791323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a5d9f-c064-4987-bfe0-1ba1cc0b2454 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.803561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 983/1958] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:05.807503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 976/1959] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:05.807633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60671480-58a9-4fb7-9f0b-6524b72b7b27 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.808450 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a5d9f-c064-4987-bfe0-1ba1cc0b2454 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.817338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60671480-58a9-4fb7-9f0b-6524b72b7b27 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.818749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a5d9f-c064-4987-bfe0-1ba1cc0b2454 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:05.823120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71e1c7-50f4-433c-b903-bc85b8d024db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.823341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954a565-de2c-4dc3-bc05-db98e9483cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:05.825824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71e1c7-50f4-433c-b903-bc85b8d024db admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.825959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954a565-de2c-4dc3-bc05-db98e9483cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:05.826057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71e1c7-50f4-433c-b903-bc85b8d024db admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.826130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954a565-de2c-4dc3-bc05-db98e9483cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:05.826208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71e1c7-50f4-433c-b903-bc85b8d024db admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.826335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954a565-de2c-4dc3-bc05-db98e9483cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:05.826463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71e1c7-50f4-433c-b903-bc85b8d024db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.826581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954a565-de2c-4dc3-bc05-db98e9483cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:05.827277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71e1c7-50f4-433c-b903-bc85b8d024db admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.827382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954a565-de2c-4dc3-bc05-db98e9483cc3 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:05.942631 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 977/1960] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:05 2022] POST /identity/v3/projects => generated 393 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:05.949303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954a565-de2c-4dc3-bc05-db98e9483cc3 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:05.971746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 984/1961] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:05 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:05.978004 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71e1c7-50f4-433c-b903-bc85b8d024db admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.016942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954a565-de2c-4dc3-bc05-db98e9483cc3 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.061468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71e1c7-50f4-433c-b903-bc85b8d024db admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.066368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc8542-9727-403c-9edf-03d2c5d3d33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.067560 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc8542-9727-403c-9edf-03d2c5d3d33f admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.067839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc8542-9727-403c-9edf-03d2c5d3d33f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.068112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc8542-9727-403c-9edf-03d2c5d3d33f admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.068825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc8542-9727-403c-9edf-03d2c5d3d33f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.070698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc8542-9727-403c-9edf-03d2c5d3d33f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.081917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821675d7-b0f1-4ca0-b50d-5d8868b33db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.083541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821675d7-b0f1-4ca0-b50d-5d8868b33db0 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.083945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821675d7-b0f1-4ca0-b50d-5d8868b33db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.084211 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821675d7-b0f1-4ca0-b50d-5d8868b33db0 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.084865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821675d7-b0f1-4ca0-b50d-5d8868b33db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.087067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821675d7-b0f1-4ca0-b50d-5d8868b33db0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.175834 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 985/1962] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:05 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:06.180140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc8542-9727-403c-9edf-03d2c5d3d33f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.186535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 978/1963] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:05 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:06.190932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc8542-9727-403c-9edf-03d2c5d3d33f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.192931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821675d7-b0f1-4ca0-b50d-5d8868b33db0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.196363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461feea1-0a6d-4bc1-881d-223374d0fcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.197025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461feea1-0a6d-4bc1-881d-223374d0fcac admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.197307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461feea1-0a6d-4bc1-881d-223374d0fcac admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.202960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461feea1-0a6d-4bc1-881d-223374d0fcac admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.202960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461feea1-0a6d-4bc1-881d-223374d0fcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.202960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461feea1-0a6d-4bc1-881d-223374d0fcac admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.203235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821675d7-b0f1-4ca0-b50d-5d8868b33db0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.208227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ed652-59e6-402c-9db5-e859d3069485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.208821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ed652-59e6-402c-9db5-e859d3069485 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.209094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ed652-59e6-402c-9db5-e859d3069485 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.209363 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ed652-59e6-402c-9db5-e859d3069485 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.210033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ed652-59e6-402c-9db5-e859d3069485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.210619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 986/1964] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:06.211621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ed652-59e6-402c-9db5-e859d3069485 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.214936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461feea1-0a6d-4bc1-881d-223374d0fcac admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.226463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 979/1965] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:06.226652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461feea1-0a6d-4bc1-881d-223374d0fcac admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.227638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ed652-59e6-402c-9db5-e859d3069485 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.229457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6ac60-455f-4585-8f9c-d4add1d65982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.230104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ac60-455f-4585-8f9c-d4add1d65982 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.230383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ac60-455f-4585-8f9c-d4add1d65982 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.230653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ac60-455f-4585-8f9c-d4add1d65982 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.231380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6ac60-455f-4585-8f9c-d4add1d65982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.232999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6ac60-455f-4585-8f9c-d4add1d65982 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.237856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ed652-59e6-402c-9db5-e859d3069485 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.242535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4cb7-8123-4ad7-8584-912b974c842e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.243129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4cb7-8123-4ad7-8584-912b974c842e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.243530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4cb7-8123-4ad7-8584-912b974c842e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.243806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4cb7-8123-4ad7-8584-912b974c842e admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.244476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc4cb7-8123-4ad7-8584-912b974c842e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.244877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 987/1966] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:06.246115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc4cb7-8123-4ad7-8584-912b974c842e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.249407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6ac60-455f-4585-8f9c-d4add1d65982 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.261867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 980/1967] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:06.262047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6ac60-455f-4585-8f9c-d4add1d65982 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.262983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4cb7-8123-4ad7-8584-912b974c842e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.265386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae063788-76ea-4951-8227-b76541ec152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.266066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae063788-76ea-4951-8227-b76541ec152b admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.266367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae063788-76ea-4951-8227-b76541ec152b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.267841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:06.268574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae063788-76ea-4951-8227-b76541ec152b admin admin] PATH_INFO: `/v3/projects/6d306ca047a542d0b3e2ae1b4dff72a2/users/79dca2b07cc84bceb8ff9b280eb16e18/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.273921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4cb7-8123-4ad7-8584-912b974c842e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.279209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85988034-c9e4-4a35-b6e2-5f04373bdf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.279747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85988034-c9e4-4a35-b6e2-5f04373bdf31 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.280056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85988034-c9e4-4a35-b6e2-5f04373bdf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.280335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85988034-c9e4-4a35-b6e2-5f04373bdf31 admin admin] PATH_INFO: `/v3/projects/95a12a4ab09647b39a2ef01ecdeaf0da/users/8042709775bd490d9e4427e60d1b8405/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.285771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae063788-76ea-4951-8227-b76541ec152b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d306ca047a542d0b3e2ae1b4dff72a2, user_id=79dca2b07cc84bceb8ff9b280eb16e18, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.289055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae063788-76ea-4951-8227-b76541ec152b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.295636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85988034-c9e4-4a35-b6e2-5f04373bdf31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a12a4ab09647b39a2ef01ecdeaf0da, user_id=8042709775bd490d9e4427e60d1b8405, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.298343 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85988034-c9e4-4a35-b6e2-5f04373bdf31 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.439873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 981/1968] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:06 2022] PUT /identity/v3/projects/95a12a4ab09647b39a2ef01ecdeaf0da/users/8042709775bd490d9e4427e60d1b8405/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:06.442841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 988/1969] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:06 2022] PUT /identity/v3/projects/6d306ca047a542d0b3e2ae1b4dff72a2/users/79dca2b07cc84bceb8ff9b280eb16e18/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:06.464426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb098a-cde7-4ee8-87be-5f3ca232078d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.464745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb098a-cde7-4ee8-87be-5f3ca232078d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.465016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb098a-cde7-4ee8-87be-5f3ca232078d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.465204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270ff42-09d4-4ab1-a082-d31e8a8ea786 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.465496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270ff42-09d4-4ab1-a082-d31e8a8ea786 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.465772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270ff42-09d4-4ab1-a082-d31e8a8ea786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.496749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3270ff42-09d4-4ab1-a082-d31e8a8ea786 None None] MFA Rules not processed for user `8042709775bd490d9e4427e60d1b8405`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:06.497385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-7afb098a-cde7-4ee8-87be-5f3ca232078d None None] MFA Rules not processed for user `79dca2b07cc84bceb8ff9b280eb16e18`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:06.498260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3270ff42-09d4-4ab1-a082-d31e8a8ea786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:06.498584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7afb098a-cde7-4ee8-87be-5f3ca232078d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:06.575122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 982/1970] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:06.576709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 989/1971] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:06.580141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb098a-cde7-4ee8-87be-5f3ca232078d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.581229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270ff42-09d4-4ab1-a082-d31e8a8ea786 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.677452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb098a-cde7-4ee8-87be-5f3ca232078d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.679801 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270ff42-09d4-4ab1-a082-d31e8a8ea786 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:06.682208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f82e2-ab99-4004-8ca6-84e90f43c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.682910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f82e2-ab99-4004-8ca6-84e90f43c051 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.683192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f82e2-ab99-4004-8ca6-84e90f43c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.683466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f82e2-ab99-4004-8ca6-84e90f43c051 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.684116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f82e2-ab99-4004-8ca6-84e90f43c051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.685428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc63b1a-7207-4a49-872c-a52c7dc4f2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:06.686079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f82e2-ab99-4004-8ca6-84e90f43c051 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.686179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc63b1a-7207-4a49-872c-a52c7dc4f2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:06.686361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc63b1a-7207-4a49-872c-a52c7dc4f2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:06.686634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc63b1a-7207-4a49-872c-a52c7dc4f2d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:06.687796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc63b1a-7207-4a49-872c-a52c7dc4f2d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:06.690028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc63b1a-7207-4a49-872c-a52c7dc4f2d9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:06.741537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:06.874598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 983/1972] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:06 2022] POST /identity/v3/projects => generated 385 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:06.879481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f82e2-ab99-4004-8ca6-84e90f43c051 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.948031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 990/1973] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:06 2022] POST /identity/v3/projects => generated 391 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:06.961271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc63b1a-7207-4a49-872c-a52c7dc4f2d9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:06.991175 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f82e2-ab99-4004-8ca6-84e90f43c051 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.101591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54a5b3-486d-465d-b9bb-4feadf89124c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.102536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54a5b3-486d-465d-b9bb-4feadf89124c admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.102990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54a5b3-486d-465d-b9bb-4feadf89124c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.103589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54a5b3-486d-465d-b9bb-4feadf89124c admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.104387 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54a5b3-486d-465d-b9bb-4feadf89124c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.105563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc63b1a-7207-4a49-872c-a52c7dc4f2d9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.107029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54a5b3-486d-465d-b9bb-4feadf89124c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.114237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5afb3-1392-45b9-a491-f1c40b8e73ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.115019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5afb3-1392-45b9-a491-f1c40b8e73ee admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.115793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5afb3-1392-45b9-a491-f1c40b8e73ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.115793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5afb3-1392-45b9-a491-f1c40b8e73ee admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.117051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5afb3-1392-45b9-a491-f1c40b8e73ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.119389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5afb3-1392-45b9-a491-f1c40b8e73ee admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.214233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 991/1974] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:06 2022] POST /identity/v3/users => generated 302 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:07.214843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 984/1975] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:06 2022] POST /identity/v3/users => generated 293 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:07.217202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54a5b3-486d-465d-b9bb-4feadf89124c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.218104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5afb3-1392-45b9-a491-f1c40b8e73ee admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.229948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5afb3-1392-45b9-a491-f1c40b8e73ee admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.230177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54a5b3-486d-465d-b9bb-4feadf89124c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.236776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dc27b-8d46-423e-b4cb-ada6ee1b8ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.237543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c90d9-344b-44e4-86ce-aede2897519a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.238328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc27b-8d46-423e-b4cb-ada6ee1b8ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.238862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc27b-8d46-423e-b4cb-ada6ee1b8ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.239018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c90d9-344b-44e4-86ce-aede2897519a admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.239441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c90d9-344b-44e4-86ce-aede2897519a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.239663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dc27b-8d46-423e-b4cb-ada6ee1b8ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.239774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c90d9-344b-44e4-86ce-aede2897519a admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.240645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dc27b-8d46-423e-b4cb-ada6ee1b8ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.241174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c90d9-344b-44e4-86ce-aede2897519a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.242921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183dc27b-8d46-423e-b4cb-ada6ee1b8ea0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.244230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c90d9-344b-44e4-86ce-aede2897519a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.255908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 992/1976] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:07.257836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 985/1977] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:07.262008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dc27b-8d46-423e-b4cb-ada6ee1b8ea0 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.263216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c90d9-344b-44e4-86ce-aede2897519a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.280775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dc27b-8d46-423e-b4cb-ada6ee1b8ea0 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.283203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c90d9-344b-44e4-86ce-aede2897519a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.286762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f54475-cadb-40cf-af3d-08895f9b207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.287597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f54475-cadb-40cf-af3d-08895f9b207a admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.288164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f54475-cadb-40cf-af3d-08895f9b207a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.288603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f54475-cadb-40cf-af3d-08895f9b207a admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.289473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f54475-cadb-40cf-af3d-08895f9b207a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.290704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e6ddb-4242-4876-95af-7a3b4530d97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.291355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f54475-cadb-40cf-af3d-08895f9b207a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.291673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e6ddb-4242-4876-95af-7a3b4530d97c admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.292014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e6ddb-4242-4876-95af-7a3b4530d97c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.292290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e6ddb-4242-4876-95af-7a3b4530d97c admin admin] PATH_INFO: `/v3/projects/927c39c5f5274ac586a8ff3507d9b2a0/users/360fd2fa94164e8e92baffdc157e0295/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.309657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:07.311145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 993/1978] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:07.313235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e6ddb-4242-4876-95af-7a3b4530d97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=927c39c5f5274ac586a8ff3507d9b2a0, user_id=360fd2fa94164e8e92baffdc157e0295, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.315266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f54475-cadb-40cf-af3d-08895f9b207a admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.316069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e6ddb-4242-4876-95af-7a3b4530d97c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.329781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f54475-cadb-40cf-af3d-08895f9b207a admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.335681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a10bb-968f-48c2-8a89-d031004e693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.336362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a10bb-968f-48c2-8a89-d031004e693f admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.336653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a10bb-968f-48c2-8a89-d031004e693f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.336916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a10bb-968f-48c2-8a89-d031004e693f admin admin] PATH_INFO: `/v3/projects/4a80fb68fdca4516ad091ef4ef4189f1/users/9d805a78cf6045e4b5ca10b0bd36c5e2/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.357262 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a10bb-968f-48c2-8a89-d031004e693f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a80fb68fdca4516ad091ef4ef4189f1, user_id=9d805a78cf6045e4b5ca10b0bd36c5e2, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.360065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a10bb-968f-48c2-8a89-d031004e693f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.409947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 986/1979] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:07 2022] PUT /identity/v3/projects/927c39c5f5274ac586a8ff3507d9b2a0/users/360fd2fa94164e8e92baffdc157e0295/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:07.409947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e6ddb-4242-4876-95af-7a3b4530d97c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.511048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e6ddb-4242-4876-95af-7a3b4530d97c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.527943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af68272-4a13-4717-8d11-cc2aadf2ffaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.528649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af68272-4a13-4717-8d11-cc2aadf2ffaa admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.528997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af68272-4a13-4717-8d11-cc2aadf2ffaa admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.529337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af68272-4a13-4717-8d11-cc2aadf2ffaa admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.530185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af68272-4a13-4717-8d11-cc2aadf2ffaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.531834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af68272-4a13-4717-8d11-cc2aadf2ffaa admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.544576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 994/1980] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:07 2022] PUT /identity/v3/projects/4a80fb68fdca4516ad091ef4ef4189f1/users/9d805a78cf6045e4b5ca10b0bd36c5e2/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:07.548358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 987/1981] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:07.559789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a10bb-968f-48c2-8a89-d031004e693f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.563678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2d41b-8fca-4d19-be21-297a48573a04 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.564045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2d41b-8fca-4d19-be21-297a48573a04 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.564405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2d41b-8fca-4d19-be21-297a48573a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.614065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-e8c2d41b-8fca-4d19-be21-297a48573a04 None None] MFA Rules not processed for user `9d805a78cf6045e4b5ca10b0bd36c5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:07.615610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e8c2d41b-8fca-4d19-be21-297a48573a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:07.656724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a10bb-968f-48c2-8a89-d031004e693f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.662440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38018848-4afe-49f8-8898-3fbc80f43c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.663780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38018848-4afe-49f8-8898-3fbc80f43c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.664034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38018848-4afe-49f8-8898-3fbc80f43c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.664296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38018848-4afe-49f8-8898-3fbc80f43c5c admin admin] PATH_INFO: `/v3/projects/927c39c5f5274ac586a8ff3507d9b2a0/users/360fd2fa94164e8e92baffdc157e0295/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.669481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38018848-4afe-49f8-8898-3fbc80f43c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=927c39c5f5274ac586a8ff3507d9b2a0, user_id=360fd2fa94164e8e92baffdc157e0295, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.675501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38018848-4afe-49f8-8898-3fbc80f43c5c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.699890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 995/1982] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:07 2022] PUT /identity/v3/projects/927c39c5f5274ac586a8ff3507d9b2a0/users/360fd2fa94164e8e92baffdc157e0295/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:07.710780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38fbff-210d-4e5f-a63b-3ab5b2cede5a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.711184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38fbff-210d-4e5f-a63b-3ab5b2cede5a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.711335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38fbff-210d-4e5f-a63b-3ab5b2cede5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.773520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:07.773520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 988/1983] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:07.773520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2d41b-8fca-4d19-be21-297a48573a04 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.774957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2d41b-8fca-4d19-be21-297a48573a04 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:07.776140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1b38fbff-210d-4e5f-a63b-3ab5b2cede5a None None] MFA Rules not processed for user `360fd2fa94164e8e92baffdc157e0295`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:07.776808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1b38fbff-210d-4e5f-a63b-3ab5b2cede5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:07.856675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d74c90-d559-4b7b-ad6e-80db546e8c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:07.856675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d74c90-d559-4b7b-ad6e-80db546e8c1d admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:07.856675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d74c90-d559-4b7b-ad6e-80db546e8c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:07.856675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d74c90-d559-4b7b-ad6e-80db546e8c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:07.857286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d74c90-d559-4b7b-ad6e-80db546e8c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:07.860317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d74c90-d559-4b7b-ad6e-80db546e8c1d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:07.906028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 996/1984] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:07 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:07.985082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 989/1985] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:07 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:07.992185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38fbff-210d-4e5f-a63b-3ab5b2cede5a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.994219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d74c90-d559-4b7b-ad6e-80db546e8c1d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:07.995921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-50d74c90-d559-4b7b-ad6e-80db546e8c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:08.005169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38fbff-210d-4e5f-a63b-3ab5b2cede5a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:08.025543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:08.103564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d74c90-d559-4b7b-ad6e-80db546e8c1d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:08.107897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bba44a-d095-42ed-82af-f53c67c6fcc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:08.108617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bba44a-d095-42ed-82af-f53c67c6fcc4 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.109074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bba44a-d095-42ed-82af-f53c67c6fcc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.109421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bba44a-d095-42ed-82af-f53c67c6fcc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.113838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8406eb-dd98-4f5a-8d0e-e17da0d41155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:08.114588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8406eb-dd98-4f5a-8d0e-e17da0d41155 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.114969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8406eb-dd98-4f5a-8d0e-e17da0d41155 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.115322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8406eb-dd98-4f5a-8d0e-e17da0d41155 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.116024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8406eb-dd98-4f5a-8d0e-e17da0d41155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:08.118171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8406eb-dd98-4f5a-8d0e-e17da0d41155 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:08.178788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bba44a-d095-42ed-82af-f53c67c6fcc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:08.180457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bba44a-d095-42ed-82af-f53c67c6fcc4 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:08.190193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 990/1986] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:07 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:08.190998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bba44a-d095-42ed-82af-f53c67c6fcc4 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:08.192829 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-46bba44a-d095-42ed-82af-f53c67c6fcc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:08.206800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bba44a-d095-42ed-82af-f53c67c6fcc4 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:08.216236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:08.216236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.216236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.217105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] PATH_INFO: `/v3/users/ec93488d852c46bb9d813ec09c7ecf28` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.218727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec93488d852c46bb9d813ec09c7ecf28)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:08.221642 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:08.257502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 997/1987] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:07 2022] POST /identity/v3/users => generated 299 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:08.260480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8406eb-dd98-4f5a-8d0e-e17da0d41155 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:08.279911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8406eb-dd98-4f5a-8d0e-e17da0d41155 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:08.283780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3599b-0f5a-4d42-9039-a57b12ea01b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:08.284583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3599b-0f5a-4d42-9039-a57b12ea01b5 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.285037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3599b-0f5a-4d42-9039-a57b12ea01b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.285471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3599b-0f5a-4d42-9039-a57b12ea01b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.364111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3599b-0f5a-4d42-9039-a57b12ea01b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:08.366432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3599b-0f5a-4d42-9039-a57b12ea01b5 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:08.385988 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 998/1988] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:08.387252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3599b-0f5a-4d42-9039-a57b12ea01b5 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:08.408809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3599b-0f5a-4d42-9039-a57b12ea01b5 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:08.420107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1c2fa-8dad-44ba-8752-6b2ed45756f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:08.421054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1c2fa-8dad-44ba-8752-6b2ed45756f9 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.421534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1c2fa-8dad-44ba-8752-6b2ed45756f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.422111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1c2fa-8dad-44ba-8752-6b2ed45756f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.423381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1c2fa-8dad-44ba-8752-6b2ed45756f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:08.426701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1c2fa-8dad-44ba-8752-6b2ed45756f9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:08.430803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec93488d852c46bb9d813ec09c7ecf28'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:08.442958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 999/1989] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:08.452343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1c2fa-8dad-44ba-8752-6b2ed45756f9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:08.454681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1c2fa-8dad-44ba-8752-6b2ed45756f9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:08.461013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a8de3-90bf-463e-a4fc-78a7ac8b94d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:08.461681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a8de3-90bf-463e-a4fc-78a7ac8b94d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.462002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a8de3-90bf-463e-a4fc-78a7ac8b94d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.462401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a8de3-90bf-463e-a4fc-78a7ac8b94d1 admin admin] PATH_INFO: `/v3/projects/8ac17f864d8246b9b496c19801f44394/users/a602b1c5ad55483b9f3e93c3b30e4205/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.477989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a8de3-90bf-463e-a4fc-78a7ac8b94d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac17f864d8246b9b496c19801f44394, user_id=a602b1c5ad55483b9f3e93c3b30e4205, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:08.480793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a8de3-90bf-463e-a4fc-78a7ac8b94d1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:08.518495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec93488d852c46bb9d813ec09c7ecf28'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:08.534214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec93488d852c46bb9d813ec09c7ecf28'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:08.544888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec93488d852c46bb9d813ec09c7ecf28'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:08.545684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-61ce01fb-b579-4047-8dde-08a3d2bc7802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec93488d852c46bb9d813ec09c7ecf28'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:08.555502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 991/1990] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:08 2022] DELETE /identity/v3/users/ec93488d852c46bb9d813ec09c7ecf28 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:08.557267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367a658-aeec-46ed-aa1c-7c44baaf1b3b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.557550 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367a658-aeec-46ed-aa1c-7c44baaf1b3b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.557845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367a658-aeec-46ed-aa1c-7c44baaf1b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.576958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-5367a658-aeec-46ed-aa1c-7c44baaf1b3b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:08.578182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5367a658-aeec-46ed-aa1c-7c44baaf1b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:08.603475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1000/1991] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:08 2022] PUT /identity/v3/projects/8ac17f864d8246b9b496c19801f44394/users/a602b1c5ad55483b9f3e93c3b30e4205/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:08.603475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a8de3-90bf-463e-a4fc-78a7ac8b94d1 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:08.603475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c64a8de3-90bf-463e-a4fc-78a7ac8b94d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:08.679457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 992/1992] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:08.689352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367a658-aeec-46ed-aa1c-7c44baaf1b3b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:08.694608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5367a658-aeec-46ed-aa1c-7c44baaf1b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:08.707742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a8de3-90bf-463e-a4fc-78a7ac8b94d1 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:08.714018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30d778-1b70-43e5-9935-340014211c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:08.714018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30d778-1b70-43e5-9935-340014211c1d admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.714018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30d778-1b70-43e5-9935-340014211c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.714347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30d778-1b70-43e5-9935-340014211c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.714968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30d778-1b70-43e5-9935-340014211c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:08.716629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30d778-1b70-43e5-9935-340014211c1d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:08.732774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1001/1993] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:08.737206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30d778-1b70-43e5-9935-340014211c1d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:08.749948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30d778-1b70-43e5-9935-340014211c1d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:08.755745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367a658-aeec-46ed-aa1c-7c44baaf1b3b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:08.755958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb74c0c-f895-42e3-a2c8-ae70e9f7dc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:08.756694 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb74c0c-f895-42e3-a2c8-ae70e9f7dc1d admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.756975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb74c0c-f895-42e3-a2c8-ae70e9f7dc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.757245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb74c0c-f895-42e3-a2c8-ae70e9f7dc1d admin admin] PATH_INFO: `/v3/projects/8ac17f864d8246b9b496c19801f44394/users/a602b1c5ad55483b9f3e93c3b30e4205/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.759437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb74c0c-f895-42e3-a2c8-ae70e9f7dc1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac17f864d8246b9b496c19801f44394, user_id=a602b1c5ad55483b9f3e93c3b30e4205, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:08.762162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb74c0c-f895-42e3-a2c8-ae70e9f7dc1d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:08.777350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1002/1994] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:08 2022] PUT /identity/v3/projects/8ac17f864d8246b9b496c19801f44394/users/a602b1c5ad55483b9f3e93c3b30e4205/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:08.796234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fe963-d9d6-410c-8341-f8ac186da3a3 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.797011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fe963-d9d6-410c-8341-f8ac186da3a3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.797312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fe963-d9d6-410c-8341-f8ac186da3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.831608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece3794-3033-4aab-8050-afb9a438063d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:08.832261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece3794-3033-4aab-8050-afb9a438063d service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:08.832627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece3794-3033-4aab-8050-afb9a438063d service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:08.832873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece3794-3033-4aab-8050-afb9a438063d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:08.839010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-fb9fe963-d9d6-410c-8341-f8ac186da3a3 None None] MFA Rules not processed for user `a602b1c5ad55483b9f3e93c3b30e4205`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:08.840230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fb9fe963-d9d6-410c-8341-f8ac186da3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:08.845196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece3794-3033-4aab-8050-afb9a438063d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:08.846661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece3794-3033-4aab-8050-afb9a438063d service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:08.857261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 993/1995] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:08.927135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1003/1996] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:08.938089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece3794-3033-4aab-8050-afb9a438063d service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:08.939944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0ece3794-3033-4aab-8050-afb9a438063d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:09.005504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece3794-3033-4aab-8050-afb9a438063d service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:09.007431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:09.079405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d325e58-2bce-4dd7-8130-b573f31ada57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:09.083014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d325e58-2bce-4dd7-8130-b573f31ada57 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:09.083014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d325e58-2bce-4dd7-8130-b573f31ada57 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:09.083014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d325e58-2bce-4dd7-8130-b573f31ada57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:09.098753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d325e58-2bce-4dd7-8130-b573f31ada57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:09.102824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d325e58-2bce-4dd7-8130-b573f31ada57 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:09.114676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 994/1997] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:09.268871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:09.521652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fe963-d9d6-410c-8341-f8ac186da3a3 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:09.524366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fb9fe963-d9d6-410c-8341-f8ac186da3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:09.688533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fe963-d9d6-410c-8341-f8ac186da3a3 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:09.695804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:09.697136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:09.698111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:09.699738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] PATH_INFO: `/v3/projects/2ea3c0d6bec144e08e216604ee25240a` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:09.700639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ea3c0d6bec144e08e216604ee25240a)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:09.702641 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:09.850139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ea3c0d6bec144e08e216604ee25240a', 'request_id': 'req-4a645e84-5512-4150-8a06-56a0af1e8a26'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:09.856655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ea3c0d6bec144e08e216604ee25240a', 'request_id': 'req-4a645e84-5512-4150-8a06-56a0af1e8a26'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:09.992335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ea3c0d6bec144e08e216604ee25240a', 'request_id': 'req-4a645e84-5512-4150-8a06-56a0af1e8a26'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:10.007274 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] The token cache is being invalidate because project 2ea3c0d6bec144e08e216604ee25240a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:10.007824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:10.008736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:10.014828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1004/1998] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:09 2022] DELETE /identity/v3/projects/2ea3c0d6bec144e08e216604ee25240a => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:10.016009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d325e58-2bce-4dd7-8130-b573f31ada57 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:10.031688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d325e58-2bce-4dd7-8130-b573f31ada57 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:10.108482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:10.109092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:10.109962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:10.110308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] PATH_INFO: `/v3/users/4067fcf33b454ae6b2f5d0829ac85c0b` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:10.111534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4067fcf33b454ae6b2f5d0829ac85c0b)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:10.114168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:10.395165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4067fcf33b454ae6b2f5d0829ac85c0b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:10.481062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4067fcf33b454ae6b2f5d0829ac85c0b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:10.502828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4067fcf33b454ae6b2f5d0829ac85c0b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:10.517616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4067fcf33b454ae6b2f5d0829ac85c0b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:10.517948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4067fcf33b454ae6b2f5d0829ac85c0b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:10.526682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 995/1999] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:10 2022] DELETE /identity/v3/users/4067fcf33b454ae6b2f5d0829ac85c0b => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:10.872050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:10.956363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:10.960066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:11.059056 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a645e84-5512-4150-8a06-56a0af1e8a26 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:11.083090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:11.085474 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:11.145642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d1da0-3b5f-4a66-bb48-ad0f040221cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:11.146310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1da0-3b5f-4a66-bb48-ad0f040221cf service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:11.146591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1da0-3b5f-4a66-bb48-ad0f040221cf service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:11.147168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1da0-3b5f-4a66-bb48-ad0f040221cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:11.162808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9c2d1da0-3b5f-4a66-bb48-ad0f040221cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:11.224546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794addf-25d6-453a-8c1b-a1d44d0142db admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:11.228908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:11.232690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:11.233448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:11.233807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:11.234240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] PATH_INFO: `/v3/projects/f113306f3a2e4c57805b1976f3500082` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:11.235424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f113306f3a2e4c57805b1976f3500082)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:11.237832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:11.243544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d1da0-3b5f-4a66-bb48-ad0f040221cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:11.245006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d1da0-3b5f-4a66-bb48-ad0f040221cf service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:11.255333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1005/2000] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:11.415630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f113306f3a2e4c57805b1976f3500082', 'request_id': 'req-bd5dbc11-ff3b-4163-bab6-e2709f668070'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:11.422969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f113306f3a2e4c57805b1976f3500082', 'request_id': 'req-bd5dbc11-ff3b-4163-bab6-e2709f668070'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:11.534063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f113306f3a2e4c57805b1976f3500082', 'request_id': 'req-bd5dbc11-ff3b-4163-bab6-e2709f668070'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:11.551359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] The token cache is being invalidate because project f113306f3a2e4c57805b1976f3500082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:11.551603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:11.552258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:11.553851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 996/2001] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:11 2022] DELETE /identity/v3/projects/f113306f3a2e4c57805b1976f3500082 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:11.558110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d1da0-3b5f-4a66-bb48-ad0f040221cf service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:11.569389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d1da0-3b5f-4a66-bb48-ad0f040221cf service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:11.628395 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:11.629914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:11.629914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:11.630459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] PATH_INFO: `/v3/users/f1eed91fd0f44405b0750d884fbb9051` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:11.631639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1eed91fd0f44405b0750d884fbb9051)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:11.633576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:11.912910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1eed91fd0f44405b0750d884fbb9051'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.027123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1eed91fd0f44405b0750d884fbb9051'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.037154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1eed91fd0f44405b0750d884fbb9051'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.050679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1eed91fd0f44405b0750d884fbb9051'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.051456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-64c2368b-2721-42c3-9b3f-62ea48d28c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1eed91fd0f44405b0750d884fbb9051'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.064708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1006/2002] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:11 2022] DELETE /identity/v3/users/f1eed91fd0f44405b0750d884fbb9051 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:12.570757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:12.572334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:12.586221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dbc11-ff3b-4163-bab6-e2709f668070 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:12.657668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:12.657668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:12.657668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:12.658215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] PATH_INFO: `/v3/projects/22f74ae8f74d457d935f63b6d5026ba1` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:12.659013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f74ae8f74d457d935f63b6d5026ba1)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:12.660968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:12.804530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f74ae8f74d457d935f63b6d5026ba1', 'request_id': 'req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.810566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f74ae8f74d457d935f63b6d5026ba1', 'request_id': 'req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.929257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f74ae8f74d457d935f63b6d5026ba1', 'request_id': 'req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.940197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] The token cache is being invalidate because project 22f74ae8f74d457d935f63b6d5026ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:12.940484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.941123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e169f2d3-dcd5-481a-b9c9-d1d56ab2438e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:12.942756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 997/2003] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:12 2022] DELETE /identity/v3/projects/22f74ae8f74d457d935f63b6d5026ba1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:12.948620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70638b-eeb1-48ae-b25e-88eff84ad437 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:12.948908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70638b-eeb1-48ae-b25e-88eff84ad437 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:12.949224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70638b-eeb1-48ae-b25e-88eff84ad437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:12.969764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ef70638b-eeb1-48ae-b25e-88eff84ad437 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:12.970993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ef70638b-eeb1-48ae-b25e-88eff84ad437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:13.033422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1007/2004] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:13.047260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e5c28-3da7-45d3-bba8-129cb4a79d17 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.047544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e5c28-3da7-45d3-bba8-129cb4a79d17 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.048069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e5c28-3da7-45d3-bba8-129cb4a79d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.068620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0b3e5c28-3da7-45d3-bba8-129cb4a79d17 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:13.069900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0b3e5c28-3da7-45d3-bba8-129cb4a79d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:13.077750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 998/2005] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:13.082542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70638b-eeb1-48ae-b25e-88eff84ad437 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:13.083133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:13.092384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70638b-eeb1-48ae-b25e-88eff84ad437 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:13.101942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50820-1ad3-4f1b-85a8-a4aebd6b3e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:13.102609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50820-1ad3-4f1b-85a8-a4aebd6b3e9b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.102886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50820-1ad3-4f1b-85a8-a4aebd6b3e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.103154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50820-1ad3-4f1b-85a8-a4aebd6b3e9b admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.103860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50820-1ad3-4f1b-85a8-a4aebd6b3e9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:13.104158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50820-1ad3-4f1b-85a8-a4aebd6b3e9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:13.105836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50820-1ad3-4f1b-85a8-a4aebd6b3e9b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:13.120405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1008/2006] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:13.131102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e5c28-3da7-45d3-bba8-129cb4a79d17 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:13.140892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e5c28-3da7-45d3-bba8-129cb4a79d17 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:13.147405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca113a0-a490-45a8-8bc3-c99690a593c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:13.147405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca113a0-a490-45a8-8bc3-c99690a593c1 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.147405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca113a0-a490-45a8-8bc3-c99690a593c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.147933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca113a0-a490-45a8-8bc3-c99690a593c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.148156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca113a0-a490-45a8-8bc3-c99690a593c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:13.150089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca113a0-a490-45a8-8bc3-c99690a593c1 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:13.153396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:13.248548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 999/2007] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:13 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:13.249813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50820-1ad3-4f1b-85a8-a4aebd6b3e9b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:13.262267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50820-1ad3-4f1b-85a8-a4aebd6b3e9b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:13.325228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce921a-3186-4a26-a046-d620cca0eff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:13.328745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce921a-3186-4a26-a046-d620cca0eff2 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.328745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce921a-3186-4a26-a046-d620cca0eff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.328745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce921a-3186-4a26-a046-d620cca0eff2 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.328745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce921a-3186-4a26-a046-d620cca0eff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:13.329203 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce921a-3186-4a26-a046-d620cca0eff2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:13.424874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1009/2008] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:13 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:13.442325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca113a0-a490-45a8-8bc3-c99690a593c1 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:13.442325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca113a0-a490-45a8-8bc3-c99690a593c1 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:13.442764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f9d6a-e422-48f1-819c-c679fb80034e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:13.443384 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f9d6a-e422-48f1-819c-c679fb80034e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.443661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f9d6a-e422-48f1-819c-c679fb80034e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.443930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f9d6a-e422-48f1-819c-c679fb80034e admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.444610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f9d6a-e422-48f1-819c-c679fb80034e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:13.446314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f9d6a-e422-48f1-819c-c679fb80034e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:13.459091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1000/2009] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:13.464834 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce921a-3186-4a26-a046-d620cca0eff2 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:13.476511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce921a-3186-4a26-a046-d620cca0eff2 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:13.481830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292af0a-1ed4-4a3a-aef5-9e3fc1b5f787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:13.482521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292af0a-1ed4-4a3a-aef5-9e3fc1b5f787 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.482804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292af0a-1ed4-4a3a-aef5-9e3fc1b5f787 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.483082 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292af0a-1ed4-4a3a-aef5-9e3fc1b5f787 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.483800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292af0a-1ed4-4a3a-aef5-9e3fc1b5f787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:13.486704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292af0a-1ed4-4a3a-aef5-9e3fc1b5f787 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:13.503333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1010/2010] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:13.507381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f9d6a-e422-48f1-819c-c679fb80034e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:13.518377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f9d6a-e422-48f1-819c-c679fb80034e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:13.529238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1911b-647d-4761-95e1-0109608ae0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:13.540980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1911b-647d-4761-95e1-0109608ae0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.540980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1911b-647d-4761-95e1-0109608ae0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.540980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1911b-647d-4761-95e1-0109608ae0cc admin admin] PATH_INFO: `/v3/projects/5d9e5bcf8fdd4f4f9a0473c461688c8d/users/f14264ffc4164b2f9a1d0c5325577007/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.551938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1911b-647d-4761-95e1-0109608ae0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9e5bcf8fdd4f4f9a0473c461688c8d, user_id=f14264ffc4164b2f9a1d0c5325577007, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:13.554807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1911b-647d-4761-95e1-0109608ae0cc admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:13.645242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1001/2011] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:13 2022] PUT /identity/v3/projects/5d9e5bcf8fdd4f4f9a0473c461688c8d/users/f14264ffc4164b2f9a1d0c5325577007/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:13.656895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa5973-85ec-4d6c-b6f5-001b88f03703 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.657208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa5973-85ec-4d6c-b6f5-001b88f03703 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.657486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa5973-85ec-4d6c-b6f5-001b88f03703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.685856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-74fa5973-85ec-4d6c-b6f5-001b88f03703 None None] MFA Rules not processed for user `f14264ffc4164b2f9a1d0c5325577007`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:13.687077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-74fa5973-85ec-4d6c-b6f5-001b88f03703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:13.759220 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1011/2012] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:13.763955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1911b-647d-4761-95e1-0109608ae0cc admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:13.828232 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1911b-647d-4761-95e1-0109608ae0cc admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:13.834063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6e4f3-edd0-486e-b611-2f5f32f7f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:13.834686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6e4f3-edd0-486e-b611-2f5f32f7f74f admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:13.835623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6e4f3-edd0-486e-b611-2f5f32f7f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:13.835623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6e4f3-edd0-486e-b611-2f5f32f7f74f admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:13.835866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6e4f3-edd0-486e-b611-2f5f32f7f74f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:13.837813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6e4f3-edd0-486e-b611-2f5f32f7f74f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:13.845903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:13.936999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1002/2013] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:13 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:13.941893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa5973-85ec-4d6c-b6f5-001b88f03703 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:13.952587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:13.957669 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa5973-85ec-4d6c-b6f5-001b88f03703 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:14.014171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe14fb-356f-4961-b02f-33c152cf1efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:14.014782 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe14fb-356f-4961-b02f-33c152cf1efd admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:14.015057 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe14fb-356f-4961-b02f-33c152cf1efd admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:14.015325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe14fb-356f-4961-b02f-33c152cf1efd admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:14.015968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe14fb-356f-4961-b02f-33c152cf1efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:14.017921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe14fb-356f-4961-b02f-33c152cf1efd admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:14.380409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1012/2014] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:13 2022] POST /identity/v3/users => generated 299 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:14.384698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6e4f3-edd0-486e-b611-2f5f32f7f74f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:14.395370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6e4f3-edd0-486e-b611-2f5f32f7f74f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:14.400179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214ee40-72dd-4df7-a134-adc80748f7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:14.400940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214ee40-72dd-4df7-a134-adc80748f7af admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:14.401213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214ee40-72dd-4df7-a134-adc80748f7af admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:14.401480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214ee40-72dd-4df7-a134-adc80748f7af admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:14.402149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214ee40-72dd-4df7-a134-adc80748f7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:14.403743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214ee40-72dd-4df7-a134-adc80748f7af admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:14.415398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1003/2015] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:14.420042 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe14fb-356f-4961-b02f-33c152cf1efd admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:14.431022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe14fb-356f-4961-b02f-33c152cf1efd admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:14.435864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871af3f-3e62-47cc-8616-051464d6d2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:14.436595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871af3f-3e62-47cc-8616-051464d6d2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:14.436874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871af3f-3e62-47cc-8616-051464d6d2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:14.437147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871af3f-3e62-47cc-8616-051464d6d2b4 admin admin] PATH_INFO: `/v3/projects/f01b7dc26b154548a9b4bdc01ced68cb/users/136b16b3204040ed952880f41f9cd11f/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:14.455602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871af3f-3e62-47cc-8616-051464d6d2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f01b7dc26b154548a9b4bdc01ced68cb, user_id=136b16b3204040ed952880f41f9cd11f, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:14.458438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871af3f-3e62-47cc-8616-051464d6d2b4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:14.543454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1013/2016] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:14 2022] PUT /identity/v3/projects/f01b7dc26b154548a9b4bdc01ced68cb/users/136b16b3204040ed952880f41f9cd11f/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:14.547544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214ee40-72dd-4df7-a134-adc80748f7af admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:14.559076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214ee40-72dd-4df7-a134-adc80748f7af admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:14.630277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84ad6d-cdbd-4728-aaef-2d2be1b32969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:14.633142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84ad6d-cdbd-4728-aaef-2d2be1b32969 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:14.633142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84ad6d-cdbd-4728-aaef-2d2be1b32969 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:14.633142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84ad6d-cdbd-4728-aaef-2d2be1b32969 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:14.633142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84ad6d-cdbd-4728-aaef-2d2be1b32969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:14.633849 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84ad6d-cdbd-4728-aaef-2d2be1b32969 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:14.645475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1004/2017] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:14.649695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:14.652776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871af3f-3e62-47cc-8616-051464d6d2b4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:14.664069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871af3f-3e62-47cc-8616-051464d6d2b4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:14.669567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b6627-dfb1-465b-9cc1-b1843f1c2920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:14.673299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b6627-dfb1-465b-9cc1-b1843f1c2920 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:14.673299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b6627-dfb1-465b-9cc1-b1843f1c2920 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:14.673299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b6627-dfb1-465b-9cc1-b1843f1c2920 admin admin] PATH_INFO: `/v3/projects/f01b7dc26b154548a9b4bdc01ced68cb/users/136b16b3204040ed952880f41f9cd11f/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:14.673299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b6627-dfb1-465b-9cc1-b1843f1c2920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f01b7dc26b154548a9b4bdc01ced68cb, user_id=136b16b3204040ed952880f41f9cd11f, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:14.676213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b6627-dfb1-465b-9cc1-b1843f1c2920 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:14.688386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1014/2018] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:14 2022] PUT /identity/v3/projects/f01b7dc26b154548a9b4bdc01ced68cb/users/136b16b3204040ed952880f41f9cd11f/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:14.700184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dfd2e-fd9d-4f0e-8338-99bb57ff04f2 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:14.719923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dfd2e-fd9d-4f0e-8338-99bb57ff04f2 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:14.719923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dfd2e-fd9d-4f0e-8338-99bb57ff04f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:14.734416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-5f1dfd2e-fd9d-4f0e-8338-99bb57ff04f2 None None] MFA Rules not processed for user `136b16b3204040ed952880f41f9cd11f`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:14.735640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5f1dfd2e-fd9d-4f0e-8338-99bb57ff04f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:14.739948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:14.807472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1005/2019] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:14.818428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b6627-dfb1-465b-9cc1-b1843f1c2920 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:14.820307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-383b6627-dfb1-465b-9cc1-b1843f1c2920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:14.877270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b6627-dfb1-465b-9cc1-b1843f1c2920 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:14.880939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db95eed-6791-4711-93f0-0d4a6edf2397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:14.881532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db95eed-6791-4711-93f0-0d4a6edf2397 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:14.881831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db95eed-6791-4711-93f0-0d4a6edf2397 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:14.882103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db95eed-6791-4711-93f0-0d4a6edf2397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:14.937910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db95eed-6791-4711-93f0-0d4a6edf2397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:14.939352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db95eed-6791-4711-93f0-0d4a6edf2397 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:14.949188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1015/2020] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:15.232739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:15.346366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:15.729942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:15.850578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:16.249383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:16.387029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:16.701671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:16.826868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:17.260850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:17.328544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dfd2e-fd9d-4f0e-8338-99bb57ff04f2 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:17.330203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5f1dfd2e-fd9d-4f0e-8338-99bb57ff04f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:17.403952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dfd2e-fd9d-4f0e-8338-99bb57ff04f2 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:17.410176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:17.411183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.411523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.411852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] PATH_INFO: `/v3/users/8042709775bd490d9e4427e60d1b8405` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.413155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8042709775bd490d9e4427e60d1b8405)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:17.415370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:17.447855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:17.642469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:17.813345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8042709775bd490d9e4427e60d1b8405'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:17.916910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8042709775bd490d9e4427e60d1b8405'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:17.923951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:17.938865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8042709775bd490d9e4427e60d1b8405'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:17.950139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8042709775bd490d9e4427e60d1b8405'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:17.953522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8042709775bd490d9e4427e60d1b8405'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:17.962146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1006/2021] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:17 2022] DELETE /identity/v3/users/8042709775bd490d9e4427e60d1b8405 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:17.968266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf098d41-ff1f-4a3c-be2c-92b81fce691f None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:17.968790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf098d41-ff1f-4a3c-be2c-92b81fce691f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:17.969272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf098d41-ff1f-4a3c-be2c-92b81fce691f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:17.990985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-bf098d41-ff1f-4a3c-be2c-92b81fce691f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:17.992187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-bf098d41-ff1f-4a3c-be2c-92b81fce691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:18.080788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1016/2022] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:18.101346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:18.103585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:18.202034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4448d6-eb63-491d-8187-ddd7dbafa98a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:18.208046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2dbfc-df2b-47da-b5e0-0700bd165846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:18.209004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2dbfc-df2b-47da-b5e0-0700bd165846 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.209448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2dbfc-df2b-47da-b5e0-0700bd165846 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.209936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2dbfc-df2b-47da-b5e0-0700bd165846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.222884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2dbfc-df2b-47da-b5e0-0700bd165846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:18.225807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2dbfc-df2b-47da-b5e0-0700bd165846 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:18.239644 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1007/2023] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:18.263389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:18.644443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:18.938684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf098d41-ff1f-4a3c-be2c-92b81fce691f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:18.938684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-bf098d41-ff1f-4a3c-be2c-92b81fce691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:18.951159 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf098d41-ff1f-4a3c-be2c-92b81fce691f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:18.956164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:18.956921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:18.957255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:18.957605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] PATH_INFO: `/v3/projects/95a12a4ab09647b39a2ef01ecdeaf0da` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:18.958853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a12a4ab09647b39a2ef01ecdeaf0da)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:18.960795 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.021341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:19.058304 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a12a4ab09647b39a2ef01ecdeaf0da', 'request_id': 'req-8801ad88-70e3-4072-9b16-119a67e62474'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.066784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a12a4ab09647b39a2ef01ecdeaf0da', 'request_id': 'req-8801ad88-70e3-4072-9b16-119a67e62474'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.123883 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a12a4ab09647b39a2ef01ecdeaf0da', 'request_id': 'req-8801ad88-70e3-4072-9b16-119a67e62474'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.137429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] The token cache is being invalidate because project 95a12a4ab09647b39a2ef01ecdeaf0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:19.137429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.137711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.142383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1017/2024] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:18 2022] DELETE /identity/v3/projects/95a12a4ab09647b39a2ef01ecdeaf0da => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:19.143552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2dbfc-df2b-47da-b5e0-0700bd165846 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:19.154334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2dbfc-df2b-47da-b5e0-0700bd165846 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:19.226123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:19.226765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:19.227042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:19.227334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] PATH_INFO: `/v3/users/360fd2fa94164e8e92baffdc157e0295` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:19.228594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=360fd2fa94164e8e92baffdc157e0295)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:19.237830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:19.424167 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '360fd2fa94164e8e92baffdc157e0295'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.455230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:19.535482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '360fd2fa94164e8e92baffdc157e0295'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.549455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '360fd2fa94164e8e92baffdc157e0295'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.556674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '360fd2fa94164e8e92baffdc157e0295'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.557399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '360fd2fa94164e8e92baffdc157e0295'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:19.563772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1008/2025] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:19 2022] DELETE /identity/v3/users/360fd2fa94164e8e92baffdc157e0295 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:19.569562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:20.072455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:20.114334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.115974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:20.137414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.139174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:20.141368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801ad88-70e3-4072-9b16-119a67e62474 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.213814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:20.247792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.248587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.248871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.249181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] PATH_INFO: `/v3/projects/927c39c5f5274ac586a8ff3507d9b2a0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.250571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=927c39c5f5274ac586a8ff3507d9b2a0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.252933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.254815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e217-fed2-474f-a769-230f09ee5f55 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.259461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea514240-487b-475c-b571-4e6a83cdd5c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.260162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea514240-487b-475c-b571-4e6a83cdd5c7 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.260445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea514240-487b-475c-b571-4e6a83cdd5c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.260720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea514240-487b-475c-b571-4e6a83cdd5c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.263052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ea514240-487b-475c-b571-4e6a83cdd5c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:20.331280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea514240-487b-475c-b571-4e6a83cdd5c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.333271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea514240-487b-475c-b571-4e6a83cdd5c7 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.343536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1009/2026] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:20.435215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927c39c5f5274ac586a8ff3507d9b2a0', 'request_id': 'req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:20.445167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927c39c5f5274ac586a8ff3507d9b2a0', 'request_id': 'req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:20.523075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927c39c5f5274ac586a8ff3507d9b2a0', 'request_id': 'req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:20.535620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] The token cache is being invalidate because project 927c39c5f5274ac586a8ff3507d9b2a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:20.536352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:20.537216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a4c44521-e78f-4b4d-8d5c-a0e78ca397b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:20.539361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1018/2027] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:20 2022] DELETE /identity/v3/projects/927c39c5f5274ac586a8ff3507d9b2a0 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:20.544573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bfec7-2ddd-413e-81ea-a91a0684ae40 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.545145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bfec7-2ddd-413e-81ea-a91a0684ae40 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.545319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bfec7-2ddd-413e-81ea-a91a0684ae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.570197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-253bfec7-2ddd-413e-81ea-a91a0684ae40 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:20.570197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-253bfec7-2ddd-413e-81ea-a91a0684ae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:20.646014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1010/2028] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:20.654397 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbcb91-6e33-4284-857f-f2c3533d5dc4 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.654967 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbcb91-6e33-4284-857f-f2c3533d5dc4 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.655492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbcb91-6e33-4284-857f-f2c3533d5dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.684346 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-abbbcb91-6e33-4284-857f-f2c3533d5dc4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:20.685714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-abbbcb91-6e33-4284-857f-f2c3533d5dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:20.708146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1019/2029] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:20.710198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bfec7-2ddd-413e-81ea-a91a0684ae40 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.724514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bfec7-2ddd-413e-81ea-a91a0684ae40 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.731231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9935c0b-7969-4777-9c88-9534855e10c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.732388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9935c0b-7969-4777-9c88-9534855e10c8 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.733008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9935c0b-7969-4777-9c88-9534855e10c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.733697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9935c0b-7969-4777-9c88-9534855e10c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.735065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9935c0b-7969-4777-9c88-9534855e10c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:20.735769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9935c0b-7969-4777-9c88-9534855e10c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.739555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9935c0b-7969-4777-9c88-9534855e10c8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.763779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1011/2030] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:20.771185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbcb91-6e33-4284-857f-f2c3533d5dc4 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.787886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbcb91-6e33-4284-857f-f2c3533d5dc4 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:20.796234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f693cd4-b1ae-47b0-9a47-03ee2a37cd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:20.796996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f693cd4-b1ae-47b0-9a47-03ee2a37cd54 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:20.798624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f693cd4-b1ae-47b0-9a47-03ee2a37cd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:20.798624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f693cd4-b1ae-47b0-9a47-03ee2a37cd54 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:20.798624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f693cd4-b1ae-47b0-9a47-03ee2a37cd54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:20.801012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f693cd4-b1ae-47b0-9a47-03ee2a37cd54 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:20.962660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1020/2031] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:20 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:20.964282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9935c0b-7969-4777-9c88-9534855e10c8 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:20.975080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9935c0b-7969-4777-9c88-9534855e10c8 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:21.046473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea68ce-7253-4299-abfb-01e8acb1c58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:21.047137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea68ce-7253-4299-abfb-01e8acb1c58c admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:21.047365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea68ce-7253-4299-abfb-01e8acb1c58c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:21.047633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea68ce-7253-4299-abfb-01e8acb1c58c admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:21.048263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea68ce-7253-4299-abfb-01e8acb1c58c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:21.050723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea68ce-7253-4299-abfb-01e8acb1c58c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:21.184162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1012/2032] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:20 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:21.188448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f693cd4-b1ae-47b0-9a47-03ee2a37cd54 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:21.199648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f693cd4-b1ae-47b0-9a47-03ee2a37cd54 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:21.206718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602f3e1-55d8-40d0-b6af-98162c6e16f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:21.207711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602f3e1-55d8-40d0-b6af-98162c6e16f4 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:21.208170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602f3e1-55d8-40d0-b6af-98162c6e16f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:21.208453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602f3e1-55d8-40d0-b6af-98162c6e16f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:21.209210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602f3e1-55d8-40d0-b6af-98162c6e16f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:21.210955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602f3e1-55d8-40d0-b6af-98162c6e16f4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:21.224154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1021/2033] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:21.228835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea68ce-7253-4299-abfb-01e8acb1c58c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:21.240409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea68ce-7253-4299-abfb-01e8acb1c58c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:21.245261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad4ec3-a9c9-4da1-8f9e-b805a36b7ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:21.245975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad4ec3-a9c9-4da1-8f9e-b805a36b7ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:21.246251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad4ec3-a9c9-4da1-8f9e-b805a36b7ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:21.246520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad4ec3-a9c9-4da1-8f9e-b805a36b7ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:21.247191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad4ec3-a9c9-4da1-8f9e-b805a36b7ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:21.248812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad4ec3-a9c9-4da1-8f9e-b805a36b7ad8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:21.266515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1013/2034] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:21.267900 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602f3e1-55d8-40d0-b6af-98162c6e16f4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:21.279574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602f3e1-55d8-40d0-b6af-98162c6e16f4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:21.289567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07198e-0fbe-4fa8-8172-4b483c1db90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:21.290677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07198e-0fbe-4fa8-8172-4b483c1db90d admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:21.291111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07198e-0fbe-4fa8-8172-4b483c1db90d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:21.291703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07198e-0fbe-4fa8-8172-4b483c1db90d admin admin] PATH_INFO: `/v3/projects/f6d81f182531461886ce8dbdca95d627/users/438619387cb547e994bc3405234b988f/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:21.312332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07198e-0fbe-4fa8-8172-4b483c1db90d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6d81f182531461886ce8dbdca95d627, user_id=438619387cb547e994bc3405234b988f, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:21.315508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07198e-0fbe-4fa8-8172-4b483c1db90d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:21.335476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:21.377138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1022/2035] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:21 2022] PUT /identity/v3/projects/f6d81f182531461886ce8dbdca95d627/users/438619387cb547e994bc3405234b988f/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:21.387779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed760c2b-6f88-4ad3-8e93-7108f64bd37a None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:21.388043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed760c2b-6f88-4ad3-8e93-7108f64bd37a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:21.388355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed760c2b-6f88-4ad3-8e93-7108f64bd37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:21.435922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:21.438913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ed760c2b-6f88-4ad3-8e93-7108f64bd37a None None] MFA Rules not processed for user `438619387cb547e994bc3405234b988f`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:21.442309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ed760c2b-6f88-4ad3-8e93-7108f64bd37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:21.529958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1014/2036] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:21.535132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07198e-0fbe-4fa8-8172-4b483c1db90d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:21.609388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07198e-0fbe-4fa8-8172-4b483c1db90d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:21.615634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3590e13-44bd-45c6-b215-08fcdf0009ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:21.616267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3590e13-44bd-45c6-b215-08fcdf0009ca admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:21.616545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3590e13-44bd-45c6-b215-08fcdf0009ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:21.616889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3590e13-44bd-45c6-b215-08fcdf0009ca admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:21.617535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3590e13-44bd-45c6-b215-08fcdf0009ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:21.619595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3590e13-44bd-45c6-b215-08fcdf0009ca admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:21.745753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1023/2037] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:21 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:21.750803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed760c2b-6f88-4ad3-8e93-7108f64bd37a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:21.771158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed760c2b-6f88-4ad3-8e93-7108f64bd37a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:21.849983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383b8d5-f0c6-4104-a6fe-9e577cb5e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:21.850506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383b8d5-f0c6-4104-a6fe-9e577cb5e906 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:21.850771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383b8d5-f0c6-4104-a6fe-9e577cb5e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:21.851034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383b8d5-f0c6-4104-a6fe-9e577cb5e906 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:21.851674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383b8d5-f0c6-4104-a6fe-9e577cb5e906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:21.854141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383b8d5-f0c6-4104-a6fe-9e577cb5e906 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:21.994170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1015/2038] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:21 2022] POST /identity/v3/users => generated 299 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:21.998120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3590e13-44bd-45c6-b215-08fcdf0009ca admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:22.009824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3590e13-44bd-45c6-b215-08fcdf0009ca admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:22.014767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1261722-5e62-441a-b97a-3a55fd9cf2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:22.015401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1261722-5e62-441a-b97a-3a55fd9cf2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:22.015679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1261722-5e62-441a-b97a-3a55fd9cf2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:22.015946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1261722-5e62-441a-b97a-3a55fd9cf2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:22.016592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1261722-5e62-441a-b97a-3a55fd9cf2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:22.018223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1261722-5e62-441a-b97a-3a55fd9cf2b6 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:22.030600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1024/2039] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:22.038768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383b8d5-f0c6-4104-a6fe-9e577cb5e906 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:22.053273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383b8d5-f0c6-4104-a6fe-9e577cb5e906 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:22.060676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b18b6-3430-402f-a2db-66e1cf8035b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:22.062782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b18b6-3430-402f-a2db-66e1cf8035b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:22.063105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b18b6-3430-402f-a2db-66e1cf8035b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:22.063377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b18b6-3430-402f-a2db-66e1cf8035b6 admin admin] PATH_INFO: `/v3/projects/c892a2974ad240f7afb84acf3c02cf67/users/9b0cefb645614a5381b30c5d444ab991/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:22.083593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b18b6-3430-402f-a2db-66e1cf8035b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c892a2974ad240f7afb84acf3c02cf67, user_id=9b0cefb645614a5381b30c5d444ab991, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:22.086988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b18b6-3430-402f-a2db-66e1cf8035b6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:22.230892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1016/2040] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:22 2022] PUT /identity/v3/projects/c892a2974ad240f7afb84acf3c02cf67/users/9b0cefb645614a5381b30c5d444ab991/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:22.240682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1261722-5e62-441a-b97a-3a55fd9cf2b6 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:22.278357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1261722-5e62-441a-b97a-3a55fd9cf2b6 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:22.324750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:22.378033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e30e0-f3ca-4f01-bd90-573529d7f6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:22.379525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e30e0-f3ca-4f01-bd90-573529d7f6ee admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:22.380015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e30e0-f3ca-4f01-bd90-573529d7f6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:22.380607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e30e0-f3ca-4f01-bd90-573529d7f6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:22.381482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e30e0-f3ca-4f01-bd90-573529d7f6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:22.383905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e30e0-f3ca-4f01-bd90-573529d7f6ee admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:22.397302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:22.405050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1025/2041] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:22.409715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b18b6-3430-402f-a2db-66e1cf8035b6 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:22.424305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b18b6-3430-402f-a2db-66e1cf8035b6 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:22.432559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa54d5b-2a32-425a-b9ae-1f65fff85124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:22.438217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54d5b-2a32-425a-b9ae-1f65fff85124 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:22.438560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54d5b-2a32-425a-b9ae-1f65fff85124 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:22.438818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54d5b-2a32-425a-b9ae-1f65fff85124 admin admin] PATH_INFO: `/v3/projects/c892a2974ad240f7afb84acf3c02cf67/users/9b0cefb645614a5381b30c5d444ab991/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:22.443758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa54d5b-2a32-425a-b9ae-1f65fff85124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c892a2974ad240f7afb84acf3c02cf67, user_id=9b0cefb645614a5381b30c5d444ab991, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:22.448312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa54d5b-2a32-425a-b9ae-1f65fff85124 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:22.464619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1017/2042] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:22 2022] PUT /identity/v3/projects/c892a2974ad240f7afb84acf3c02cf67/users/9b0cefb645614a5381b30c5d444ab991/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:22.475337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb7e9c-9dc9-4275-8204-82f0a1dee5f9 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:22.475725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb7e9c-9dc9-4275-8204-82f0a1dee5f9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:22.475887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb7e9c-9dc9-4275-8204-82f0a1dee5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:22.513186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6bfb7e9c-9dc9-4275-8204-82f0a1dee5f9 None None] MFA Rules not processed for user `9b0cefb645614a5381b30c5d444ab991`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:22.514547 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6bfb7e9c-9dc9-4275-8204-82f0a1dee5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:22.620091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1026/2043] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:22.791462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa54d5b-2a32-425a-b9ae-1f65fff85124 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:22.796958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-efa54d5b-2a32-425a-b9ae-1f65fff85124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:22.887634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa54d5b-2a32-425a-b9ae-1f65fff85124 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:22.893759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44174cb9-1955-475e-aea6-f13d66e31b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:22.894994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44174cb9-1955-475e-aea6-f13d66e31b75 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:22.895600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44174cb9-1955-475e-aea6-f13d66e31b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:22.896297 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44174cb9-1955-475e-aea6-f13d66e31b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:22.930022 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44174cb9-1955-475e-aea6-f13d66e31b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:22.930022 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44174cb9-1955-475e-aea6-f13d66e31b75 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:22.944350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1018/2044] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:23.204071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:23.256908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb7e9c-9dc9-4275-8204-82f0a1dee5f9 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:23.258516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6bfb7e9c-9dc9-4275-8204-82f0a1dee5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:23.329341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb7e9c-9dc9-4275-8204-82f0a1dee5f9 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:23.336025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:23.336695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:23.336969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:23.337284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] PATH_INFO: `/v3/users/0d8f7df0786d4008a96dcae5c03ec8d5` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:23.339214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d8f7df0786d4008a96dcae5c03ec8d5)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:23.341252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:23.354298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:23.614359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d8f7df0786d4008a96dcae5c03ec8d5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:23.694555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d8f7df0786d4008a96dcae5c03ec8d5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:23.702513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d8f7df0786d4008a96dcae5c03ec8d5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:23.715391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d8f7df0786d4008a96dcae5c03ec8d5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:23.716139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d8f7df0786d4008a96dcae5c03ec8d5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:23.722680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1027/2045] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:23 2022] DELETE /identity/v3/users/0d8f7df0786d4008a96dcae5c03ec8d5 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:23.727786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e890b9-1aef-410a-b114-e5cae226eb26 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:23.730034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e890b9-1aef-410a-b114-e5cae226eb26 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:23.730410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e890b9-1aef-410a-b114-e5cae226eb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:23.749949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-e4e890b9-1aef-410a-b114-e5cae226eb26 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:23.751062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e4e890b9-1aef-410a-b114-e5cae226eb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:23.824629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1019/2046] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:23.835562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:23.836943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:23.899682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a354117-aba3-4f1c-9cef-9a0c48792aa8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:23.903230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394316d8-460d-4c52-9d31-db32d4efe376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:23.903837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394316d8-460d-4c52-9d31-db32d4efe376 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:23.904119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394316d8-460d-4c52-9d31-db32d4efe376 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:23.904458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394316d8-460d-4c52-9d31-db32d4efe376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:23.914728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394316d8-460d-4c52-9d31-db32d4efe376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:23.916199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394316d8-460d-4c52-9d31-db32d4efe376 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:23.926755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1028/2047] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:24.191865 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:24.306653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:24.314312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e890b9-1aef-410a-b114-e5cae226eb26 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:24.316964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e4e890b9-1aef-410a-b114-e5cae226eb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:24.389591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e890b9-1aef-410a-b114-e5cae226eb26 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:24.397064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201afd3-85aa-4e79-b29d-2d85dbcc3c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:24.397834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201afd3-85aa-4e79-b29d-2d85dbcc3c26 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:24.397992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201afd3-85aa-4e79-b29d-2d85dbcc3c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:24.398256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201afd3-85aa-4e79-b29d-2d85dbcc3c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:24.400100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d201afd3-85aa-4e79-b29d-2d85dbcc3c26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:24.465909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201afd3-85aa-4e79-b29d-2d85dbcc3c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:24.467287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201afd3-85aa-4e79-b29d-2d85dbcc3c26 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:24.477133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1020/2048] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:24.527296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394316d8-460d-4c52-9d31-db32d4efe376 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:24.529334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-394316d8-460d-4c52-9d31-db32d4efe376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:24.568646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394316d8-460d-4c52-9d31-db32d4efe376 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:24.575209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:24.576468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:24.577115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:24.577637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] PATH_INFO: `/v3/projects/8d2885294fa1446782ac71b679d9d1e7` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:24.579356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d2885294fa1446782ac71b679d9d1e7)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:24.581466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:24.709033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d2885294fa1446782ac71b679d9d1e7', 'request_id': 'req-5389a53c-2164-4ecc-aba1-b84bf25de397'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:24.723201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d2885294fa1446782ac71b679d9d1e7', 'request_id': 'req-5389a53c-2164-4ecc-aba1-b84bf25de397'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:24.854563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d2885294fa1446782ac71b679d9d1e7', 'request_id': 'req-5389a53c-2164-4ecc-aba1-b84bf25de397'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:24.868244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] The token cache is being invalidate because project 8d2885294fa1446782ac71b679d9d1e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:24.868541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:24.869322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:24.871150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1029/2049] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:24 2022] DELETE /identity/v3/projects/8d2885294fa1446782ac71b679d9d1e7 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:24.875394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201afd3-85aa-4e79-b29d-2d85dbcc3c26 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:24.886370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201afd3-85aa-4e79-b29d-2d85dbcc3c26 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:24.941974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:24.942594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:24.942870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:24.943136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] PATH_INFO: `/v3/users/7ca738fea6a0471d9eecc084a322598d` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:24.944293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca738fea6a0471d9eecc084a322598d)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:24.946370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:25.311278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca738fea6a0471d9eecc084a322598d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.420524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:25.422301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:25.436036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389a53c-2164-4ecc-aba1-b84bf25de397 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:25.441175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:25.441908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:25.442184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:25.442450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] PATH_INFO: `/v3/users/438619387cb547e994bc3405234b988f` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:25.443557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] RBAC: Authorizing `identity:delete_user(user_id=438619387cb547e994bc3405234b988f)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:25.451002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:25.464160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca738fea6a0471d9eecc084a322598d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.479329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca738fea6a0471d9eecc084a322598d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.487752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca738fea6a0471d9eecc084a322598d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.488560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-006557fe-915b-45dd-9aa2-41427aeaa926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca738fea6a0471d9eecc084a322598d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.495296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1021/2050] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:24 2022] DELETE /identity/v3/users/7ca738fea6a0471d9eecc084a322598d => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:25.723746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438619387cb547e994bc3405234b988f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.792089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:25.832234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438619387cb547e994bc3405234b988f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.841229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438619387cb547e994bc3405234b988f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.859567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438619387cb547e994bc3405234b988f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.860362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438619387cb547e994bc3405234b988f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:25.866691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1030/2051] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:25 2022] DELETE /identity/v3/users/438619387cb547e994bc3405234b988f => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:25.871504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200d001-bd6b-4d18-a1bc-797b49447baf None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:25.871847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200d001-bd6b-4d18-a1bc-797b49447baf None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:25.872133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200d001-bd6b-4d18-a1bc-797b49447baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:25.892441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-5200d001-bd6b-4d18-a1bc-797b49447baf None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:25.894165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5200d001-bd6b-4d18-a1bc-797b49447baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:25.984772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1022/2052] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:25.998932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.000333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.009857 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:26.034209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200d001-bd6b-4d18-a1bc-797b49447baf None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.035975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5200d001-bd6b-4d18-a1bc-797b49447baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.056548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200d001-bd6b-4d18-a1bc-797b49447baf None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.064932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.066372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.066795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.067014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] PATH_INFO: `/v3/users/79dca2b07cc84bceb8ff9b280eb16e18` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.068699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79dca2b07cc84bceb8ff9b280eb16e18)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.071039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.079743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a271-f460-418c-9c4a-e36f00a696df admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.083673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31a7b8-5e79-4d2f-a240-43c115c2a0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.084225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31a7b8-5e79-4d2f-a240-43c115c2a0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.084497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31a7b8-5e79-4d2f-a240-43c115c2a0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.084761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31a7b8-5e79-4d2f-a240-43c115c2a0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.094339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31a7b8-5e79-4d2f-a240-43c115c2a0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.096077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31a7b8-5e79-4d2f-a240-43c115c2a0d2 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.107070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1031/2053] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:26.107954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31a7b8-5e79-4d2f-a240-43c115c2a0d2 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.109320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2f31a7b8-5e79-4d2f-a240-43c115c2a0d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.132135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31a7b8-5e79-4d2f-a240-43c115c2a0d2 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.137077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.137717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.138043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.138310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] PATH_INFO: `/v3/projects/deb0c73a1f7942a99bfbbb6d49b3ba05` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.139450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb0c73a1f7942a99bfbbb6d49b3ba05)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.141361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.344066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb0c73a1f7942a99bfbbb6d49b3ba05', 'request_id': 'req-5c033661-eb10-455c-a993-375a8de9db8c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.347436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79dca2b07cc84bceb8ff9b280eb16e18'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.354882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb0c73a1f7942a99bfbbb6d49b3ba05', 'request_id': 'req-5c033661-eb10-455c-a993-375a8de9db8c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.452109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb0c73a1f7942a99bfbbb6d49b3ba05', 'request_id': 'req-5c033661-eb10-455c-a993-375a8de9db8c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.473660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] The token cache is being invalidate because project deb0c73a1f7942a99bfbbb6d49b3ba05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:26.473660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.473660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5c033661-eb10-455c-a993-375a8de9db8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.477281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1032/2054] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:26 2022] DELETE /identity/v3/projects/deb0c73a1f7942a99bfbbb6d49b3ba05 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:26.479324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41294644-427c-4850-a740-c3edf410b890 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.479678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41294644-427c-4850-a740-c3edf410b890 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.480020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41294644-427c-4850-a740-c3edf410b890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.510261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-41294644-427c-4850-a740-c3edf410b890 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:26.512632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-41294644-427c-4850-a740-c3edf410b890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.522831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79dca2b07cc84bceb8ff9b280eb16e18'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.540977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79dca2b07cc84bceb8ff9b280eb16e18'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.554143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79dca2b07cc84bceb8ff9b280eb16e18'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.554924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1bac1400-3a0b-489c-a3ca-181831a1ebf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79dca2b07cc84bceb8ff9b280eb16e18'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.561043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1023/2055] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:26 2022] DELETE /identity/v3/users/79dca2b07cc84bceb8ff9b280eb16e18 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:26.571030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03013440-7a8d-4e2b-8d00-5ad5a2556aac None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.571312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03013440-7a8d-4e2b-8d00-5ad5a2556aac None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.571583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03013440-7a8d-4e2b-8d00-5ad5a2556aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.617424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-03013440-7a8d-4e2b-8d00-5ad5a2556aac None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:26.617424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-03013440-7a8d-4e2b-8d00-5ad5a2556aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.620545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1033/2056] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:26.621613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41294644-427c-4850-a740-c3edf410b890 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.629656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-41294644-427c-4850-a740-c3edf410b890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.641468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41294644-427c-4850-a740-c3edf410b890 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.684750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1024/2057] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:26.686133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1367-918d-4b90-87ee-84a92de4062a None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.686426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1367-918d-4b90-87ee-84a92de4062a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.686719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1367-918d-4b90-87ee-84a92de4062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.701089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.701724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.702035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.702306 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] PATH_INFO: `/v3/projects/f6d81f182531461886ce8dbdca95d627` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.703449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6d81f182531461886ce8dbdca95d627)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.705531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.707996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0a2f1367-918d-4b90-87ee-84a92de4062a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:26.708968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0a2f1367-918d-4b90-87ee-84a92de4062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.717428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1025/2058] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:26.718200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f1367-918d-4b90-87ee-84a92de4062a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.719426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0a2f1367-918d-4b90-87ee-84a92de4062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:26.778863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f1367-918d-4b90-87ee-84a92de4062a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.782498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ca552-491f-429f-855b-4bbb64d36244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.783094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ca552-491f-429f-855b-4bbb64d36244 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.783372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ca552-491f-429f-855b-4bbb64d36244 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.783641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ca552-491f-429f-855b-4bbb64d36244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.793698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ca552-491f-429f-855b-4bbb64d36244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.795235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ca552-491f-429f-855b-4bbb64d36244 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.804652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1026/2059] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:26.805546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ca552-491f-429f-855b-4bbb64d36244 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.815562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ca552-491f-429f-855b-4bbb64d36244 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.820678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218244e-1988-41f4-971d-0c8432077dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.821272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218244e-1988-41f4-971d-0c8432077dfb admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.821551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218244e-1988-41f4-971d-0c8432077dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.821869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218244e-1988-41f4-971d-0c8432077dfb admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.822537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218244e-1988-41f4-971d-0c8432077dfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:26.822860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218244e-1988-41f4-971d-0c8432077dfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.824530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218244e-1988-41f4-971d-0c8432077dfb admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.837519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1027/2060] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:26.841569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218244e-1988-41f4-971d-0c8432077dfb admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:26.852709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218244e-1988-41f4-971d-0c8432077dfb admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:26.868564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6d81f182531461886ce8dbdca95d627', 'request_id': 'req-9c397988-22f9-479b-88f9-a191e3d32c19'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.875587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6d81f182531461886ce8dbdca95d627', 'request_id': 'req-9c397988-22f9-479b-88f9-a191e3d32c19'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.915857 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563e3ae-660c-4ebb-b163-e25cf96c5efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:26.916480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563e3ae-660c-4ebb-b163-e25cf96c5efa admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:26.917017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563e3ae-660c-4ebb-b163-e25cf96c5efa admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:26.917284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563e3ae-660c-4ebb-b163-e25cf96c5efa admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:26.917948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563e3ae-660c-4ebb-b163-e25cf96c5efa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:26.919883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563e3ae-660c-4ebb-b163-e25cf96c5efa admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:26.975352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6d81f182531461886ce8dbdca95d627', 'request_id': 'req-9c397988-22f9-479b-88f9-a191e3d32c19'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.991592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] The token cache is being invalidate because project f6d81f182531461886ce8dbdca95d627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:26.991592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.992699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:26.996152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1034/2061] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:26 2022] DELETE /identity/v3/projects/f6d81f182531461886ce8dbdca95d627 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:27.000071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.002419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.019195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:27.025402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1028/2062] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:26 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:27.026824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563e3ae-660c-4ebb-b163-e25cf96c5efa admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.030142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7563e3ae-660c-4ebb-b163-e25cf96c5efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.037107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c397988-22f9-479b-88f9-a191e3d32c19 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.117734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563e3ae-660c-4ebb-b163-e25cf96c5efa admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.123892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.124835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.125147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.125869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] PATH_INFO: `/v3/users/9b0cefb645614a5381b30c5d444ab991` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.127605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0cefb645614a5381b30c5d444ab991)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.130730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.133143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5e9a2-6c39-43a5-b8d2-5c7c457a5c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.138182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5e9a2-6c39-43a5-b8d2-5c7c457a5c12 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.138182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5e9a2-6c39-43a5-b8d2-5c7c457a5c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.138182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5e9a2-6c39-43a5-b8d2-5c7c457a5c12 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.138182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5e9a2-6c39-43a5-b8d2-5c7c457a5c12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.138182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5e9a2-6c39-43a5-b8d2-5c7c457a5c12 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.182340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:27.235506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1029/2063] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:27 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:27.240759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5e9a2-6c39-43a5-b8d2-5c7c457a5c12 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.261765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5e9a2-6c39-43a5-b8d2-5c7c457a5c12 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.269289 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cd959-e11c-463d-9bbe-8e5b6bf23d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.270165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd959-e11c-463d-9bbe-8e5b6bf23d9e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.270698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd959-e11c-463d-9bbe-8e5b6bf23d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.271184 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd959-e11c-463d-9bbe-8e5b6bf23d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.272400 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cd959-e11c-463d-9bbe-8e5b6bf23d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.275729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cd959-e11c-463d-9bbe-8e5b6bf23d9e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.291397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1030/2064] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:27.296163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cd959-e11c-463d-9bbe-8e5b6bf23d9e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.308367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cd959-e11c-463d-9bbe-8e5b6bf23d9e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.313505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f706f4-e756-4b8d-a789-1bfe4cc3120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.314150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f706f4-e756-4b8d-a789-1bfe4cc3120f admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.314425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f706f4-e756-4b8d-a789-1bfe4cc3120f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.314692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f706f4-e756-4b8d-a789-1bfe4cc3120f admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.316222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f706f4-e756-4b8d-a789-1bfe4cc3120f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.317867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f706f4-e756-4b8d-a789-1bfe4cc3120f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.334240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1031/2065] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:27.337286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0cefb645614a5381b30c5d444ab991'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.338697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f706f4-e756-4b8d-a789-1bfe4cc3120f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.354925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f706f4-e756-4b8d-a789-1bfe4cc3120f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.360282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8021e6e-e041-4f06-94cf-408fcaf30a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.360873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021e6e-e041-4f06-94cf-408fcaf30a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.361222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021e6e-e041-4f06-94cf-408fcaf30a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.361569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021e6e-e041-4f06-94cf-408fcaf30a41 admin admin] PATH_INFO: `/v3/projects/5a98c038b4604cd080b9bb4ce05f86f0/users/b4d97d270dd547b19227df86f6b4ad39/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.387719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8021e6e-e041-4f06-94cf-408fcaf30a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a98c038b4604cd080b9bb4ce05f86f0, user_id=b4d97d270dd547b19227df86f6b4ad39, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.390789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8021e6e-e041-4f06-94cf-408fcaf30a41 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.437094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0cefb645614a5381b30c5d444ab991'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.446261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0cefb645614a5381b30c5d444ab991'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.464244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0cefb645614a5381b30c5d444ab991'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.465080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0cefb645614a5381b30c5d444ab991'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.476743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1035/2066] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:26 2022] DELETE /identity/v3/users/9b0cefb645614a5381b30c5d444ab991 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:27.476743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.479637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.501855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1032/2067] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:27 2022] PUT /identity/v3/projects/5a98c038b4604cd080b9bb4ce05f86f0/users/b4d97d270dd547b19227df86f6b4ad39/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:27.513758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d60a8-e69d-4e39-9c81-97a136175747 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.514096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d60a8-e69d-4e39-9c81-97a136175747 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.514373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d60a8-e69d-4e39-9c81-97a136175747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.574473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2e1d60a8-e69d-4e39-9c81-97a136175747 None None] MFA Rules not processed for user `b4d97d270dd547b19227df86f6b4ad39`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:27.576791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2e1d60a8-e69d-4e39-9c81-97a136175747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.631048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308fd87-642e-48a2-84c8-0035fa5bba06 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.648382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.650324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.652838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.653707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] PATH_INFO: `/v3/projects/6d306ca047a542d0b3e2ae1b4dff72a2` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.656545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d306ca047a542d0b3e2ae1b4dff72a2)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.663226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.709367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1033/2068] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:27.714075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d60a8-e69d-4e39-9c81-97a136175747 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.715677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2e1d60a8-e69d-4e39-9c81-97a136175747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.731219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d60a8-e69d-4e39-9c81-97a136175747 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.735976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce3c44-5478-4ae3-bb6b-80dca1d7da55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.736610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce3c44-5478-4ae3-bb6b-80dca1d7da55 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.736888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce3c44-5478-4ae3-bb6b-80dca1d7da55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.737329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce3c44-5478-4ae3-bb6b-80dca1d7da55 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.738097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce3c44-5478-4ae3-bb6b-80dca1d7da55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.739972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce3c44-5478-4ae3-bb6b-80dca1d7da55 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:27.791880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d306ca047a542d0b3e2ae1b4dff72a2', 'request_id': 'req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.799923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d306ca047a542d0b3e2ae1b4dff72a2', 'request_id': 'req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.855471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1034/2069] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:27 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:27.859597 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce3c44-5478-4ae3-bb6b-80dca1d7da55 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.906232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d306ca047a542d0b3e2ae1b4dff72a2', 'request_id': 'req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.917991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] The token cache is being invalidate because project 6d306ca047a542d0b3e2ae1b4dff72a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:27.918310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.918962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:27.920793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1036/2070] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:27 2022] DELETE /identity/v3/projects/6d306ca047a542d0b3e2ae1b4dff72a2 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:27.924485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:27.926002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:27.929490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce3c44-5478-4ae3-bb6b-80dca1d7da55 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:27.934998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cf59f-3adb-4d75-ae97-fb95ec631d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:27.937388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cf59f-3adb-4d75-ae97-fb95ec631d5c admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:27.937388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cf59f-3adb-4d75-ae97-fb95ec631d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:27.937388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cf59f-3adb-4d75-ae97-fb95ec631d5c admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:27.937388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cf59f-3adb-4d75-ae97-fb95ec631d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:27.939473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cf59f-3adb-4d75-ae97-fb95ec631d5c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.001363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e1f47-395f-4c0a-8c73-5cbe0e71afa2 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.007048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.007685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.007984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.008259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] PATH_INFO: `/v3/users/9d805a78cf6045e4b5ca10b0bd36c5e2` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.009409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d805a78cf6045e4b5ca10b0bd36c5e2)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.011392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.050860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1035/2071] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:27 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:28.052221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cf59f-3adb-4d75-ae97-fb95ec631d5c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.054598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-701cf59f-3adb-4d75-ae97-fb95ec631d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:28.070459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cf59f-3adb-4d75-ae97-fb95ec631d5c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.075849 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.076729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.077007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.077275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] PATH_INFO: `/v3/projects/c892a2974ad240f7afb84acf3c02cf67` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.078478 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c892a2974ad240f7afb84acf3c02cf67)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.080590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.136902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:28.208013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:28.275261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d805a78cf6045e4b5ca10b0bd36c5e2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.276314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c892a2974ad240f7afb84acf3c02cf67', 'request_id': 'req-864f50ff-99f0-4a40-873c-bfe2a6e70076'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.284217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c892a2974ad240f7afb84acf3c02cf67', 'request_id': 'req-864f50ff-99f0-4a40-873c-bfe2a6e70076'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.369511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d805a78cf6045e4b5ca10b0bd36c5e2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.377812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d805a78cf6045e4b5ca10b0bd36c5e2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.393023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d805a78cf6045e4b5ca10b0bd36c5e2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.393683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d805a78cf6045e4b5ca10b0bd36c5e2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.400284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1037/2072] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:27 2022] DELETE /identity/v3/users/9d805a78cf6045e4b5ca10b0bd36c5e2 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:28.401336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.402783 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:28.457981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c892a2974ad240f7afb84acf3c02cf67', 'request_id': 'req-864f50ff-99f0-4a40-873c-bfe2a6e70076'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.473535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] The token cache is being invalidate because project c892a2974ad240f7afb84acf3c02cf67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:28.474007 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.474615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62aa82-9031-4016-85df-5c19566839f9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.474810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-864f50ff-99f0-4a40-873c-bfe2a6e70076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:28.480117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269796d7-1679-4320-9b64-95b2db6a01c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.480536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1036/2073] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:28 2022] DELETE /identity/v3/projects/c892a2974ad240f7afb84acf3c02cf67 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:28.480886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269796d7-1679-4320-9b64-95b2db6a01c4 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.481179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269796d7-1679-4320-9b64-95b2db6a01c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.481453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269796d7-1679-4320-9b64-95b2db6a01c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.482163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269796d7-1679-4320-9b64-95b2db6a01c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.482845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4f08d-6c14-439e-8c7c-3008288f0497 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.483297 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4f08d-6c14-439e-8c7c-3008288f0497 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.483777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269796d7-1679-4320-9b64-95b2db6a01c4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.484034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4f08d-6c14-439e-8c7c-3008288f0497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.496120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1038/2074] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:28.499682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269796d7-1679-4320-9b64-95b2db6a01c4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.502183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-269796d7-1679-4320-9b64-95b2db6a01c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:28.509426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a8a4f08d-6c14-439e-8c7c-3008288f0497 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:28.511080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a8a4f08d-6c14-439e-8c7c-3008288f0497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:28.515671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269796d7-1679-4320-9b64-95b2db6a01c4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.520987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1037/2075] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:28.522192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37862cf4-bb32-458c-8d42-aecfc095f17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.523040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37862cf4-bb32-458c-8d42-aecfc095f17a admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.523396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37862cf4-bb32-458c-8d42-aecfc095f17a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.523738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37862cf4-bb32-458c-8d42-aecfc095f17a admin admin] PATH_INFO: `/v3/projects/555b1476e1f54a1eab5c3641df63f0bc/users/a88a3b04c9be4b83b4bab1b0af3b6bae/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.528767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77baa2e-0338-4072-ba09-3cdfb9503b2c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.529207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77baa2e-0338-4072-ba09-3cdfb9503b2c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.529589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77baa2e-0338-4072-ba09-3cdfb9503b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.536579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:28.543719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37862cf4-bb32-458c-8d42-aecfc095f17a admin admin] RBAC: Authorizing `identity:create_grant(project_id=555b1476e1f54a1eab5c3641df63f0bc, user_id=a88a3b04c9be4b83b4bab1b0af3b6bae, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.546565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37862cf4-bb32-458c-8d42-aecfc095f17a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.553378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a77baa2e-0338-4072-ba09-3cdfb9503b2c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:28.554641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a77baa2e-0338-4072-ba09-3cdfb9503b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:28.568856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1038/2076] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:28.570724 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77baa2e-0338-4072-ba09-3cdfb9503b2c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.580182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77baa2e-0338-4072-ba09-3cdfb9503b2c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.585066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428da79-66b5-4185-93dd-d86c9219c1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.585740 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428da79-66b5-4185-93dd-d86c9219c1da admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.586048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428da79-66b5-4185-93dd-d86c9219c1da admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.586316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428da79-66b5-4185-93dd-d86c9219c1da admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.587123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428da79-66b5-4185-93dd-d86c9219c1da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:28.587429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428da79-66b5-4185-93dd-d86c9219c1da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.589138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428da79-66b5-4185-93dd-d86c9219c1da admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.605494 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1039/2077] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:01:28.609581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428da79-66b5-4185-93dd-d86c9219c1da admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.625006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428da79-66b5-4185-93dd-d86c9219c1da admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.625763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1039/2078] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:28 2022] PUT /identity/v3/projects/555b1476e1f54a1eab5c3641df63f0bc/users/a88a3b04c9be4b83b4bab1b0af3b6bae/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:28.629957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37862cf4-bb32-458c-8d42-aecfc095f17a admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.701889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37862cf4-bb32-458c-8d42-aecfc095f17a admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.707372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e378a4-0cbf-449c-b4c2-b0a1bf3700b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.707988 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e378a4-0cbf-449c-b4c2-b0a1bf3700b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.708466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e378a4-0cbf-449c-b4c2-b0a1bf3700b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.708760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e378a4-0cbf-449c-b4c2-b0a1bf3700b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.709396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e378a4-0cbf-449c-b4c2-b0a1bf3700b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.711259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e378a4-0cbf-449c-b4c2-b0a1bf3700b0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.712135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dc5ec-b611-4127-8b68-7af5ac80f532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.712843 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dc5ec-b611-4127-8b68-7af5ac80f532 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.713720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dc5ec-b611-4127-8b68-7af5ac80f532 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.714276 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dc5ec-b611-4127-8b68-7af5ac80f532 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.714868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85dc5ec-b611-4127-8b68-7af5ac80f532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.718461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85dc5ec-b611-4127-8b68-7af5ac80f532 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.723555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1040/2079] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:28.727369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e378a4-0cbf-449c-b4c2-b0a1bf3700b0 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.738236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e378a4-0cbf-449c-b4c2-b0a1bf3700b0 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.744986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf92f9b-3856-4a01-9b89-6d4fd600401f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.745666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf92f9b-3856-4a01-9b89-6d4fd600401f admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.745993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf92f9b-3856-4a01-9b89-6d4fd600401f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.746373 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf92f9b-3856-4a01-9b89-6d4fd600401f admin admin] PATH_INFO: `/v3/projects/555b1476e1f54a1eab5c3641df63f0bc/users/a88a3b04c9be4b83b4bab1b0af3b6bae/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.748362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf92f9b-3856-4a01-9b89-6d4fd600401f admin admin] RBAC: Authorizing `identity:create_grant(project_id=555b1476e1f54a1eab5c3641df63f0bc, user_id=a88a3b04c9be4b83b4bab1b0af3b6bae, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.751183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf92f9b-3856-4a01-9b89-6d4fd600401f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.762313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1041/2080] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:28 2022] PUT /identity/v3/projects/555b1476e1f54a1eab5c3641df63f0bc/users/a88a3b04c9be4b83b4bab1b0af3b6bae/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:28.772846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c5dec-c1bb-4d1c-bf3f-c7ce11eb3eb0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.773132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c5dec-c1bb-4d1c-bf3f-c7ce11eb3eb0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.773406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c5dec-c1bb-4d1c-bf3f-c7ce11eb3eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.804634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ae3c5dec-c1bb-4d1c-bf3f-c7ce11eb3eb0 None None] MFA Rules not processed for user `a88a3b04c9be4b83b4bab1b0af3b6bae`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:28.805919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ae3c5dec-c1bb-4d1c-bf3f-c7ce11eb3eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:28.851207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1040/2081] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:28 2022] POST /identity/v3/projects => generated 373 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:28.855423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dc5ec-b611-4127-8b68-7af5ac80f532 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.868909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:28.888110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1042/2082] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:28.899336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c5dec-c1bb-4d1c-bf3f-c7ce11eb3eb0 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:28.900891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ae3c5dec-c1bb-4d1c-bf3f-c7ce11eb3eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:28.926659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dc5ec-b611-4127-8b68-7af5ac80f532 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:28.929915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9afc89-efef-4e63-b344-ba3aa7128b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:28.930527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9afc89-efef-4e63-b344-ba3aa7128b10 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:28.930796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9afc89-efef-4e63-b344-ba3aa7128b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:28.931073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9afc89-efef-4e63-b344-ba3aa7128b10 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:28.931738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9afc89-efef-4e63-b344-ba3aa7128b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:28.933858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9afc89-efef-4e63-b344-ba3aa7128b10 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:28.961748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c5dec-c1bb-4d1c-bf3f-c7ce11eb3eb0 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.017052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41bde7-46a1-4eeb-8745-2590fbb70577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.021619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41bde7-46a1-4eeb-8745-2590fbb70577 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.021619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41bde7-46a1-4eeb-8745-2590fbb70577 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.021619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41bde7-46a1-4eeb-8745-2590fbb70577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.067475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1041/2083] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:28 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:29.067475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9afc89-efef-4e63-b344-ba3aa7128b10 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.067475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5c9afc89-efef-4e63-b344-ba3aa7128b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:29.081306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9afc89-efef-4e63-b344-ba3aa7128b10 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.086476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.087203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.087569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.087850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] PATH_INFO: `/v3/projects/4a80fb68fdca4516ad091ef4ef4189f1` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.090552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a80fb68fdca4516ad091ef4ef4189f1)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.091427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.097166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41bde7-46a1-4eeb-8745-2590fbb70577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.099685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41bde7-46a1-4eeb-8745-2590fbb70577 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.116360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1043/2084] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:29.119286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41bde7-46a1-4eeb-8745-2590fbb70577 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.128334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41bde7-46a1-4eeb-8745-2590fbb70577 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.133458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ae29f-d16c-4bdd-a970-0ccafe584395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.134178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ae29f-d16c-4bdd-a970-0ccafe584395 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.134516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ae29f-d16c-4bdd-a970-0ccafe584395 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.134847 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ae29f-d16c-4bdd-a970-0ccafe584395 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.135559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ae29f-d16c-4bdd-a970-0ccafe584395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.137430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ae29f-d16c-4bdd-a970-0ccafe584395 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.148341 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1044/2085] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:29.152485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ae29f-d16c-4bdd-a970-0ccafe584395 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.163183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ae29f-d16c-4bdd-a970-0ccafe584395 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.168556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba575-dc5c-43d2-b96a-59386f8259a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.169172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba575-dc5c-43d2-b96a-59386f8259a9 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.169453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba575-dc5c-43d2-b96a-59386f8259a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.169720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba575-dc5c-43d2-b96a-59386f8259a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.170386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ba575-dc5c-43d2-b96a-59386f8259a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.171973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ba575-dc5c-43d2-b96a-59386f8259a9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.183127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1045/2086] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:29.187249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba575-dc5c-43d2-b96a-59386f8259a9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.197963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba575-dc5c-43d2-b96a-59386f8259a9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.202609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc8d2f-8b2a-4850-899c-c9af33b24ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.203271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc8d2f-8b2a-4850-899c-c9af33b24ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.203548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc8d2f-8b2a-4850-899c-c9af33b24ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.203820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc8d2f-8b2a-4850-899c-c9af33b24ef8 admin admin] PATH_INFO: `/v3/projects/0a44fdc86b5a478aacdaa15226070714/users/e49512a1665b41089ad21583d4331b06/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.220174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc8d2f-8b2a-4850-899c-c9af33b24ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a44fdc86b5a478aacdaa15226070714, user_id=e49512a1665b41089ad21583d4331b06, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.222929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc8d2f-8b2a-4850-899c-c9af33b24ef8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.242879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a80fb68fdca4516ad091ef4ef4189f1', 'request_id': 'req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.249733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:29.251906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a80fb68fdca4516ad091ef4ef4189f1', 'request_id': 'req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.348409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1046/2087] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:29 2022] PUT /identity/v3/projects/0a44fdc86b5a478aacdaa15226070714/users/e49512a1665b41089ad21583d4331b06/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:29.349827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d409-c35f-4f7c-8494-730e3e4b26bc None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.349827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d409-c35f-4f7c-8494-730e3e4b26bc None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.350170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d409-c35f-4f7c-8494-730e3e4b26bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.358018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a80fb68fdca4516ad091ef4ef4189f1', 'request_id': 'req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.371071 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] The token cache is being invalidate because project 4a80fb68fdca4516ad091ef4ef4189f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:29.371710 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.372084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.373704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1042/2088] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:29 2022] DELETE /identity/v3/projects/4a80fb68fdca4516ad091ef4ef4189f1 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:29.381588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.384020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:29.386814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f762d409-c35f-4f7c-8494-730e3e4b26bc None None] MFA Rules not processed for user `e49512a1665b41089ad21583d4331b06`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:29.388025 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f762d409-c35f-4f7c-8494-730e3e4b26bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:29.464947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1047/2089] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:29.478591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9a713-8b7d-4cde-adc3-378c80d2e6da admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.500833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d409-c35f-4f7c-8494-730e3e4b26bc None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.502550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f762d409-c35f-4f7c-8494-730e3e4b26bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:29.552535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:29.558124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.558886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.559257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.559628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] PATH_INFO: `/v3/users/a602b1c5ad55483b9f3e93c3b30e4205` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.560887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a602b1c5ad55483b9f3e93c3b30e4205)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.562987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.578793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d409-c35f-4f7c-8494-730e3e4b26bc None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.590096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30031a02-58cf-448c-9110-6f36f6003304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.590096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30031a02-58cf-448c-9110-6f36f6003304 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.590096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30031a02-58cf-448c-9110-6f36f6003304 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.590096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30031a02-58cf-448c-9110-6f36f6003304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.609690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30031a02-58cf-448c-9110-6f36f6003304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.611259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30031a02-58cf-448c-9110-6f36f6003304 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.622386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1048/2090] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:29.657189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30031a02-58cf-448c-9110-6f36f6003304 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.659372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-30031a02-58cf-448c-9110-6f36f6003304 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:29.679347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30031a02-58cf-448c-9110-6f36f6003304 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.691376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.692324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.692600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.692910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] PATH_INFO: `/v3/users/e49512a1665b41089ad21583d4331b06` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.694144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49512a1665b41089ad21583d4331b06)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.696060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.783421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a602b1c5ad55483b9f3e93c3b30e4205'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.854681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a602b1c5ad55483b9f3e93c3b30e4205'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.867053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a602b1c5ad55483b9f3e93c3b30e4205'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.873980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a602b1c5ad55483b9f3e93c3b30e4205'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.874680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a602b1c5ad55483b9f3e93c3b30e4205'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:29.880664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1043/2091] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:29 2022] DELETE /identity/v3/users/a602b1c5ad55483b9f3e93c3b30e4205 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:29.881441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:29.882798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:29.948830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2524f3c-b095-4932-b4b7-be836fc3044b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:29.953485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:29.954149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:29.954436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:29.954707 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] PATH_INFO: `/v3/users/b4d97d270dd547b19227df86f6b4ad39` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:29.956541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d97d270dd547b19227df86f6b4ad39)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:29.958615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:29.984054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49512a1665b41089ad21583d4331b06'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.134960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49512a1665b41089ad21583d4331b06'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.144321 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49512a1665b41089ad21583d4331b06'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.158487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49512a1665b41089ad21583d4331b06'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.159388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5b3f703a-f33d-4fa3-b3ce-44491e0d0004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49512a1665b41089ad21583d4331b06'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.166293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1049/2092] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:29 2022] DELETE /identity/v3/users/e49512a1665b41089ad21583d4331b06 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:30.171166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248cf06-fe08-47be-adcf-de5d2de2396d None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.171735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248cf06-fe08-47be-adcf-de5d2de2396d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.172110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248cf06-fe08-47be-adcf-de5d2de2396d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.202436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c248cf06-fe08-47be-adcf-de5d2de2396d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:30.203700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c248cf06-fe08-47be-adcf-de5d2de2396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:30.257608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d97d270dd547b19227df86f6b4ad39'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.270933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1050/2093] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:30.280735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248cf06-fe08-47be-adcf-de5d2de2396d None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:30.282126 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c248cf06-fe08-47be-adcf-de5d2de2396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:30.342324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248cf06-fe08-47be-adcf-de5d2de2396d None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:30.345429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d97d270dd547b19227df86f6b4ad39'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.346832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcee28-7f6f-490b-829b-e4a82f779d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:30.347523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcee28-7f6f-490b-829b-e4a82f779d3a service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.347789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcee28-7f6f-490b-829b-e4a82f779d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.348168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcee28-7f6f-490b-829b-e4a82f779d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.362026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcee28-7f6f-490b-829b-e4a82f779d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:30.362026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcee28-7f6f-490b-829b-e4a82f779d3a service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:30.364611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d97d270dd547b19227df86f6b4ad39'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.373874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1051/2094] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:30.374813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcee28-7f6f-490b-829b-e4a82f779d3a service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:30.376777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6bfcee28-7f6f-490b-829b-e4a82f779d3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:30.377317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d97d270dd547b19227df86f6b4ad39'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.379241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-057ad8e2-d48e-465d-81ab-454bcb96f3f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d97d270dd547b19227df86f6b4ad39'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.390947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1044/2095] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:29 2022] DELETE /identity/v3/users/b4d97d270dd547b19227df86f6b4ad39 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:30.393138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcee28-7f6f-490b-829b-e4a82f779d3a service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:30.395062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79e77-0187-446c-9385-93312ffeba77 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.395346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79e77-0187-446c-9385-93312ffeba77 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.395614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79e77-0187-446c-9385-93312ffeba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.425610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-78f79e77-0187-446c-9385-93312ffeba77 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:30.427889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-78f79e77-0187-446c-9385-93312ffeba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:30.473169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:30.474403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.474869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.475227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] PATH_INFO: `/v3/projects/8ac17f864d8246b9b496c19801f44394` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.476555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac17f864d8246b9b496c19801f44394)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:30.479797 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:30.522145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1045/2096] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:30.529695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:30.535368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f79e77-0187-446c-9385-93312ffeba77 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:30.536794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-78f79e77-0187-446c-9385-93312ffeba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:30.610438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f79e77-0187-446c-9385-93312ffeba77 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:30.616392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550e96c-fb3e-4d6d-b55c-a43592a3fe1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:30.617156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550e96c-fb3e-4d6d-b55c-a43592a3fe1b service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.617449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550e96c-fb3e-4d6d-b55c-a43592a3fe1b service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.617924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550e96c-fb3e-4d6d-b55c-a43592a3fe1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.629558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550e96c-fb3e-4d6d-b55c-a43592a3fe1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:30.631817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550e96c-fb3e-4d6d-b55c-a43592a3fe1b service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:30.644151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1046/2097] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:30.683806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac17f864d8246b9b496c19801f44394', 'request_id': 'req-114f0175-74f5-4f67-9209-88cfd23754b1'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.691151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac17f864d8246b9b496c19801f44394', 'request_id': 'req-114f0175-74f5-4f67-9209-88cfd23754b1'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.819696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac17f864d8246b9b496c19801f44394', 'request_id': 'req-114f0175-74f5-4f67-9209-88cfd23754b1'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.832388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] The token cache is being invalidate because project 8ac17f864d8246b9b496c19801f44394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:30.833229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.833414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-114f0175-74f5-4f67-9209-88cfd23754b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:30.835507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1052/2098] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:30 2022] DELETE /identity/v3/projects/8ac17f864d8246b9b496c19801f44394 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:30.841489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a685f-0394-469d-a5b1-7d63eb2f4140 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.841772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a685f-0394-469d-a5b1-7d63eb2f4140 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.842072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a685f-0394-469d-a5b1-7d63eb2f4140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:30.878088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-034a685f-0394-469d-a5b1-7d63eb2f4140 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:30.879670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-034a685f-0394-469d-a5b1-7d63eb2f4140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:30.963785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1047/2099] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:30.976638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018a846-4e83-482b-9e03-73611b95a8a5 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:30.979452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:30.980092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018a846-4e83-482b-9e03-73611b95a8a5 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:30.980510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018a846-4e83-482b-9e03-73611b95a8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.047188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3018a846-4e83-482b-9e03-73611b95a8a5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:31.047800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3018a846-4e83-482b-9e03-73611b95a8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.061864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1053/2100] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:31.063924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a685f-0394-469d-a5b1-7d63eb2f4140 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.076282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a685f-0394-469d-a5b1-7d63eb2f4140 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.083584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.084172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.084921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.085193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.085965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:31.086181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.088014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.101822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1048/2101] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:31.107241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018a846-4e83-482b-9e03-73611b95a8a5 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.118260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018a846-4e83-482b-9e03-73611b95a8a5 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.125740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a7153-88ca-4332-8f4a-bb853599301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.126952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a7153-88ca-4332-8f4a-bb853599301e admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.127399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a7153-88ca-4332-8f4a-bb853599301e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.127775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a7153-88ca-4332-8f4a-bb853599301e admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.128432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a7153-88ca-4332-8f4a-bb853599301e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.130376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a7153-88ca-4332-8f4a-bb853599301e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.240224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.242755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.265711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da55da-e4af-44ff-84e0-ea68c74e2654 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.272218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.272833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.273107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.273377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] PATH_INFO: `/v3/projects/0a44fdc86b5a478aacdaa15226070714` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.274553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a44fdc86b5a478aacdaa15226070714)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.276484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.324145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1054/2102] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:31 2022] POST /identity/v3/projects => generated 397 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:31.328423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a7153-88ca-4332-8f4a-bb853599301e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.404394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a7153-88ca-4332-8f4a-bb853599301e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.418654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b317c47-d9ca-4923-bd0c-ab44667f8485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.419246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b317c47-d9ca-4923-bd0c-ab44667f8485 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.419530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b317c47-d9ca-4923-bd0c-ab44667f8485 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.419806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b317c47-d9ca-4923-bd0c-ab44667f8485 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.420443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b317c47-d9ca-4923-bd0c-ab44667f8485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.426806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b317c47-d9ca-4923-bd0c-ab44667f8485 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.426967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a44fdc86b5a478aacdaa15226070714', 'request_id': 'req-97032165-a3af-40fe-8373-4e6a4ba5053c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.434335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a44fdc86b5a478aacdaa15226070714', 'request_id': 'req-97032165-a3af-40fe-8373-4e6a4ba5053c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.523433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a44fdc86b5a478aacdaa15226070714', 'request_id': 'req-97032165-a3af-40fe-8373-4e6a4ba5053c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.528745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1055/2103] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:31 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:31.529755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b317c47-d9ca-4923-bd0c-ab44667f8485 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.534707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9b317c47-d9ca-4923-bd0c-ab44667f8485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.542814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] The token cache is being invalidate because project 0a44fdc86b5a478aacdaa15226070714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:31.543381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.544064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.548941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1049/2104] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:31 2022] DELETE /identity/v3/projects/0a44fdc86b5a478aacdaa15226070714 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:31.548941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.549868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.555730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b317c47-d9ca-4923-bd0c-ab44667f8485 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.633395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97032165-a3af-40fe-8373-4e6a4ba5053c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.638518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489ca2d-3286-49a0-b4b3-c1411285e02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.639284 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ca2d-3286-49a0-b4b3-c1411285e02e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.639565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ca2d-3286-49a0-b4b3-c1411285e02e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.639834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ca2d-3286-49a0-b4b3-c1411285e02e admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.640613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489ca2d-3286-49a0-b4b3-c1411285e02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.642404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489ca2d-3286-49a0-b4b3-c1411285e02e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.644575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.645453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.646044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.646181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] PATH_INFO: `/v3/projects/5a98c038b4604cd080b9bb4ce05f86f0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.647914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a98c038b4604cd080b9bb4ce05f86f0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.650482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.660977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1050/2105] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:31.662483 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651baca-d15e-424b-9fe3-60b1fb6c5116 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.662780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651baca-d15e-424b-9fe3-60b1fb6c5116 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.667627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651baca-d15e-424b-9fe3-60b1fb6c5116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.691971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c651baca-d15e-424b-9fe3-60b1fb6c5116 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:31.693189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c651baca-d15e-424b-9fe3-60b1fb6c5116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.701259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1051/2106] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:31.702164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651baca-d15e-424b-9fe3-60b1fb6c5116 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.712829 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651baca-d15e-424b-9fe3-60b1fb6c5116 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.717928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48a806-2608-4987-ae7e-dcd476752ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.718523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48a806-2608-4987-ae7e-dcd476752ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.718798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48a806-2608-4987-ae7e-dcd476752ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.719060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48a806-2608-4987-ae7e-dcd476752ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.719696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48a806-2608-4987-ae7e-dcd476752ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.721287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48a806-2608-4987-ae7e-dcd476752ff5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.731800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1052/2107] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:31.733154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d625df-a186-4757-a5ad-cf2d9fa52e51 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.733435 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d625df-a186-4757-a5ad-cf2d9fa52e51 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.733701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d625df-a186-4757-a5ad-cf2d9fa52e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.753140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-55d625df-a186-4757-a5ad-cf2d9fa52e51 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:31.754288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-55d625df-a186-4757-a5ad-cf2d9fa52e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.762919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1053/2108] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:31.764058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d625df-a186-4757-a5ad-cf2d9fa52e51 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.777552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d625df-a186-4757-a5ad-cf2d9fa52e51 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.788688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a98c038b4604cd080b9bb4ce05f86f0', 'request_id': 'req-c0ec52ea-f948-4ef9-91f4-057bc89494d9'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.797449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a98c038b4604cd080b9bb4ce05f86f0', 'request_id': 'req-c0ec52ea-f948-4ef9-91f4-057bc89494d9'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.855426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6f481-ced2-4dea-a810-eed6ad88d375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.856103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6f481-ced2-4dea-a810-eed6ad88d375 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.856377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6f481-ced2-4dea-a810-eed6ad88d375 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.856645 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6f481-ced2-4dea-a810-eed6ad88d375 admin admin] PATH_INFO: `/v3/projects/afc23e61e878460f98ecff82663fa6f5/users/c481e9bf83a24ef498e251c4286e1195/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.874696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6f481-ced2-4dea-a810-eed6ad88d375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc23e61e878460f98ecff82663fa6f5, user_id=c481e9bf83a24ef498e251c4286e1195, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.878311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6f481-ced2-4dea-a810-eed6ad88d375 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.894954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a98c038b4604cd080b9bb4ce05f86f0', 'request_id': 'req-c0ec52ea-f948-4ef9-91f4-057bc89494d9'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.911211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] The token cache is being invalidate because project 5a98c038b4604cd080b9bb4ce05f86f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:31.911509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.912228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:31.913984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1056/2109] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:31 2022] DELETE /identity/v3/projects/5a98c038b4604cd080b9bb4ce05f86f0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:31.918301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.918301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.931964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec52ea-f948-4ef9-91f4-057bc89494d9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.937322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:31.937992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:31.938277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:31.938544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:31.939685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:31.939685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:31.942099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:31.957473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1057/2110] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:31.959018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:31.962141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:31.989381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ef2a9-28dc-4cef-9f4d-d0ff9872d848 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:31.994820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1054/2111] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:31 2022] PUT /identity/v3/projects/afc23e61e878460f98ecff82663fa6f5/users/c481e9bf83a24ef498e251c4286e1195/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:31.996574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6f481-ced2-4dea-a810-eed6ad88d375 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.024659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:32.079078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.080173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.080630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.081589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] PATH_INFO: `/v3/users/a88a3b04c9be4b83b4bab1b0af3b6bae` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.083719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a88a3b04c9be4b83b4bab1b0af3b6bae)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.087780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.092814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6f481-ced2-4dea-a810-eed6ad88d375 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.100409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35205fd-3f61-4c6d-bdbe-6aab79ed2338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.102754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35205fd-3f61-4c6d-bdbe-6aab79ed2338 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.102754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35205fd-3f61-4c6d-bdbe-6aab79ed2338 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.102754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35205fd-3f61-4c6d-bdbe-6aab79ed2338 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.103526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35205fd-3f61-4c6d-bdbe-6aab79ed2338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.107359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35205fd-3f61-4c6d-bdbe-6aab79ed2338 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.241421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1055/2112] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:31 2022] POST /identity/v3/projects => generated 377 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:32.242825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35205fd-3f61-4c6d-bdbe-6aab79ed2338 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.245476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f35205fd-3f61-4c6d-bdbe-6aab79ed2338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.324866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35205fd-3f61-4c6d-bdbe-6aab79ed2338 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.329884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.330607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.330834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.331107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] PATH_INFO: `/v3/users/f14264ffc4164b2f9a1d0c5325577007` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.332379 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f14264ffc4164b2f9a1d0c5325577007)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.334292 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.349943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a88a3b04c9be4b83b4bab1b0af3b6bae'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.459275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a88a3b04c9be4b83b4bab1b0af3b6bae'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.467510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a88a3b04c9be4b83b4bab1b0af3b6bae'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.482521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a88a3b04c9be4b83b4bab1b0af3b6bae'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.483654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ced78cd0-9093-4768-ae00-45c18ce3669d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a88a3b04c9be4b83b4bab1b0af3b6bae'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.492690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1058/2113] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:31 2022] DELETE /identity/v3/users/a88a3b04c9be4b83b4bab1b0af3b6bae => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:32.494024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d1a96-c4da-479e-9ec3-71c851b2d795 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.494289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d1a96-c4da-479e-9ec3-71c851b2d795 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.494717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d1a96-c4da-479e-9ec3-71c851b2d795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.527120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-477d1a96-c4da-479e-9ec3-71c851b2d795 None None] MFA Rules not processed for user `c481e9bf83a24ef498e251c4286e1195`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:32.528436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-477d1a96-c4da-479e-9ec3-71c851b2d795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.598378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f14264ffc4164b2f9a1d0c5325577007'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.613771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1059/2114] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:32.614632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d1a96-c4da-479e-9ec3-71c851b2d795 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.616797 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-477d1a96-c4da-479e-9ec3-71c851b2d795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.671141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f14264ffc4164b2f9a1d0c5325577007'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.683114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d1a96-c4da-479e-9ec3-71c851b2d795 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.687113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f14264ffc4164b2f9a1d0c5325577007'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.688001 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071027b6-cf74-457b-b8b2-cc186d6f9608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.688613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071027b6-cf74-457b-b8b2-cc186d6f9608 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.688916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071027b6-cf74-457b-b8b2-cc186d6f9608 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.689181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071027b6-cf74-457b-b8b2-cc186d6f9608 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.689832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071027b6-cf74-457b-b8b2-cc186d6f9608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.691757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071027b6-cf74-457b-b8b2-cc186d6f9608 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.694468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f14264ffc4164b2f9a1d0c5325577007'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.695231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f14264ffc4164b2f9a1d0c5325577007'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:32.702844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1056/2115] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:32 2022] DELETE /identity/v3/users/f14264ffc4164b2f9a1d0c5325577007 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:32.703569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.705181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.775540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:32.784986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1060/2116] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:32 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:32.788324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea62435-5359-4479-a2df-f0dac85063c8 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.788608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea62435-5359-4479-a2df-f0dac85063c8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.788877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea62435-5359-4479-a2df-f0dac85063c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.808253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76b57e-26df-4546-813e-daafb4f7f1c6 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.814293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efaff4-c257-4125-8b16-bc6ca3e98ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.815233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efaff4-c257-4125-8b16-bc6ca3e98ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.815688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efaff4-c257-4125-8b16-bc6ca3e98ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.816105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efaff4-c257-4125-8b16-bc6ca3e98ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.817987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efaff4-c257-4125-8b16-bc6ca3e98ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.820238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efaff4-c257-4125-8b16-bc6ca3e98ff0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.833312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8ea62435-5359-4479-a2df-f0dac85063c8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:32.839228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8ea62435-5359-4479-a2df-f0dac85063c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.848723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1061/2117] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:32.850644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea62435-5359-4479-a2df-f0dac85063c8 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.853163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8ea62435-5359-4479-a2df-f0dac85063c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.867088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea62435-5359-4479-a2df-f0dac85063c8 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:32.874878 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14417fb7-7649-4747-a0f1-bbda9aa82adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:32.876216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14417fb7-7649-4747-a0f1-bbda9aa82adc admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:32.876616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14417fb7-7649-4747-a0f1-bbda9aa82adc admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:32.877426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14417fb7-7649-4747-a0f1-bbda9aa82adc admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:32.880438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14417fb7-7649-4747-a0f1-bbda9aa82adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:32.882475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14417fb7-7649-4747-a0f1-bbda9aa82adc admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:32.894344 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1062/2118] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:32.898204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14417fb7-7649-4747-a0f1-bbda9aa82adc admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.900440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-14417fb7-7649-4747-a0f1-bbda9aa82adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:32.938251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1057/2119] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:32 2022] POST /identity/v3/projects => generated 399 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:32.938251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efaff4-c257-4125-8b16-bc6ca3e98ff0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:32.965316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14417fb7-7649-4747-a0f1-bbda9aa82adc admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.025396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efaff4-c257-4125-8b16-bc6ca3e98ff0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.027590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05560b-98ca-4e21-b19b-696081568459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.028287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05560b-98ca-4e21-b19b-696081568459 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.028567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05560b-98ca-4e21-b19b-696081568459 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.028864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05560b-98ca-4e21-b19b-696081568459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.033807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b245b3f-95b7-4cd8-b5f4-e18b1c087631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.034967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b245b3f-95b7-4cd8-b5f4-e18b1c087631 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.035521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b245b3f-95b7-4cd8-b5f4-e18b1c087631 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.035521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b245b3f-95b7-4cd8-b5f4-e18b1c087631 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.036150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b245b3f-95b7-4cd8-b5f4-e18b1c087631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.037992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b245b3f-95b7-4cd8-b5f4-e18b1c087631 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.041627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05560b-98ca-4e21-b19b-696081568459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.043128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05560b-98ca-4e21-b19b-696081568459 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.070532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1063/2120] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:33.070731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1058/2121] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:33.070881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05560b-98ca-4e21-b19b-696081568459 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.070992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b245b3f-95b7-4cd8-b5f4-e18b1c087631 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.071245 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7b245b3f-95b7-4cd8-b5f4-e18b1c087631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:33.083199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05560b-98ca-4e21-b19b-696081568459 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.089070 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b245b3f-95b7-4cd8-b5f4-e18b1c087631 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.090548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e341b-d8d4-465d-b92d-ad3a7ce004c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.091583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e341b-d8d4-465d-b92d-ad3a7ce004c3 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.092067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e341b-d8d4-465d-b92d-ad3a7ce004c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.093478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e341b-d8d4-465d-b92d-ad3a7ce004c3 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.093478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e341b-d8d4-465d-b92d-ad3a7ce004c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.099034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e341b-d8d4-465d-b92d-ad3a7ce004c3 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.099428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.099428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.099428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.099428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] PATH_INFO: `/v3/projects/555b1476e1f54a1eab5c3641df63f0bc` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.101612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] RBAC: Authorizing `identity:delete_project(project_id=555b1476e1f54a1eab5c3641df63f0bc)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.104700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.185986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1064/2122] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:33 2022] POST /identity/v3/users => generated 306 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:33.186892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e341b-d8d4-465d-b92d-ad3a7ce004c3 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.199006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e341b-d8d4-465d-b92d-ad3a7ce004c3 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.206531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2ea2d-06b5-4910-bbfc-2247f4898f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.207215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2ea2d-06b5-4910-bbfc-2247f4898f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.208115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2ea2d-06b5-4910-bbfc-2247f4898f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.208115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2ea2d-06b5-4910-bbfc-2247f4898f90 admin admin] PATH_INFO: `/v3/projects/86fd78c53bab4d77bc3dc99e60559bf7/users/114587faee234dc38b709cb6e1d1de3a/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.226553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2ea2d-06b5-4910-bbfc-2247f4898f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86fd78c53bab4d77bc3dc99e60559bf7, user_id=114587faee234dc38b709cb6e1d1de3a, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.229337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2ea2d-06b5-4910-bbfc-2247f4898f90 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.235137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:33.328171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1065/2123] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:33 2022] PUT /identity/v3/projects/86fd78c53bab4d77bc3dc99e60559bf7/users/114587faee234dc38b709cb6e1d1de3a/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:33.329531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2ea2d-06b5-4910-bbfc-2247f4898f90 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.341120 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '555b1476e1f54a1eab5c3641df63f0bc', 'request_id': 'req-7843efb4-a1b3-4c48-85b7-5435c5da738f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.350855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '555b1476e1f54a1eab5c3641df63f0bc', 'request_id': 'req-7843efb4-a1b3-4c48-85b7-5435c5da738f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.383581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:33.457928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2ea2d-06b5-4910-bbfc-2247f4898f90 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.480151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '555b1476e1f54a1eab5c3641df63f0bc', 'request_id': 'req-7843efb4-a1b3-4c48-85b7-5435c5da738f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.502769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] The token cache is being invalidate because project 555b1476e1f54a1eab5c3641df63f0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:33.503246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.504327 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7843efb4-a1b3-4c48-85b7-5435c5da738f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.507057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1059/2124] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:33 2022] DELETE /identity/v3/projects/555b1476e1f54a1eab5c3641df63f0bc => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:33.508613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28c67a-6c79-4246-8a47-bcc4b1ea27a4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.509048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28c67a-6c79-4246-8a47-bcc4b1ea27a4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.509468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28c67a-6c79-4246-8a47-bcc4b1ea27a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.552750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0c28c67a-6c79-4246-8a47-bcc4b1ea27a4 None None] MFA Rules not processed for user `114587faee234dc38b709cb6e1d1de3a`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:33.554068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0c28c67a-6c79-4246-8a47-bcc4b1ea27a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:33.560971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49008d3c-662c-4ec3-bc39-ea452ce31090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.561593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49008d3c-662c-4ec3-bc39-ea452ce31090 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.562264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49008d3c-662c-4ec3-bc39-ea452ce31090 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.562781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49008d3c-662c-4ec3-bc39-ea452ce31090 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.564090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49008d3c-662c-4ec3-bc39-ea452ce31090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.565868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49008d3c-662c-4ec3-bc39-ea452ce31090 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.576378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1066/2125] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:33.577639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d0b00-a53a-457b-90d1-033f3a2d61c6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.578080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d0b00-a53a-457b-90d1-033f3a2d61c6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.578272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d0b00-a53a-457b-90d1-033f3a2d61c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.605614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1d7d0b00-a53a-457b-90d1-033f3a2d61c6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:33.607489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1d7d0b00-a53a-457b-90d1-033f3a2d61c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:33.620474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1067/2126] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:33.626077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d0b00-a53a-457b-90d1-033f3a2d61c6 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.629744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1d7d0b00-a53a-457b-90d1-033f3a2d61c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:33.646964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d0b00-a53a-457b-90d1-033f3a2d61c6 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.650604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1060/2127] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:33.650604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adab33-4342-48ea-be51-17d1355bb35f None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.650604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adab33-4342-48ea-be51-17d1355bb35f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.650604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adab33-4342-48ea-be51-17d1355bb35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.657281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1547af9-67c9-465b-b9b7-18bd311bbbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.660753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1547af9-67c9-465b-b9b7-18bd311bbbb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.660753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1547af9-67c9-465b-b9b7-18bd311bbbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.660753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1547af9-67c9-465b-b9b7-18bd311bbbb8 admin admin] PATH_INFO: `/v3/projects/e460fc409df74d5c95a427c5e962ec01/users/ac9103a951b5496c9901bf5719ec44f6/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.673249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d3adab33-4342-48ea-be51-17d1355bb35f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:33.675427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d3adab33-4342-48ea-be51-17d1355bb35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:33.689726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1061/2128] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:33.689726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adab33-4342-48ea-be51-17d1355bb35f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.689726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d3adab33-4342-48ea-be51-17d1355bb35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:33.692131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1547af9-67c9-465b-b9b7-18bd311bbbb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e460fc409df74d5c95a427c5e962ec01, user_id=ac9103a951b5496c9901bf5719ec44f6, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.695315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1547af9-67c9-465b-b9b7-18bd311bbbb8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.700215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adab33-4342-48ea-be51-17d1355bb35f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.707780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.708646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.709346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.709724 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] PATH_INFO: `/v3/projects/5d9e5bcf8fdd4f4f9a0473c461688c8d` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.711065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9e5bcf8fdd4f4f9a0473c461688c8d)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:33.713144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:33.808986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1068/2129] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:33 2022] PUT /identity/v3/projects/e460fc409df74d5c95a427c5e962ec01/users/ac9103a951b5496c9901bf5719ec44f6/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:33.812952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1547af9-67c9-465b-b9b7-18bd311bbbb8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.812952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e1547af9-67c9-465b-b9b7-18bd311bbbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:33.855335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9e5bcf8fdd4f4f9a0473c461688c8d', 'request_id': 'req-fc9dccae-c19e-4f37-bd70-d75aa71f9719'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.862606 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9e5bcf8fdd4f4f9a0473c461688c8d', 'request_id': 'req-fc9dccae-c19e-4f37-bd70-d75aa71f9719'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.876822 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1547af9-67c9-465b-b9b7-18bd311bbbb8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:33.941393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3e2c4-0a1d-4ede-8b12-775ee9b2e60b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:33.942044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3e2c4-0a1d-4ede-8b12-775ee9b2e60b service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:33.942319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3e2c4-0a1d-4ede-8b12-775ee9b2e60b service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:33.942583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3e2c4-0a1d-4ede-8b12-775ee9b2e60b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:33.945504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9e5bcf8fdd4f4f9a0473c461688c8d', 'request_id': 'req-fc9dccae-c19e-4f37-bd70-d75aa71f9719'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.960952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] The token cache is being invalidate because project 5d9e5bcf8fdd4f4f9a0473c461688c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:33.961391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.962255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:33.964814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1062/2130] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:33 2022] DELETE /identity/v3/projects/5d9e5bcf8fdd4f4f9a0473c461688c8d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:33.967839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:33.967839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.032785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3e2c4-0a1d-4ede-8b12-775ee9b2e60b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.035441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3e2c4-0a1d-4ede-8b12-775ee9b2e60b service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.045228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dccae-c19e-4f37-bd70-d75aa71f9719 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.048602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1069/2131] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:34.051215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3e2c4-0a1d-4ede-8b12-775ee9b2e60b service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.052896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:34.053999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.054719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.055493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.055861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.056580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:34.060207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.060207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.073129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1063/2132] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:34.074376 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.076262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.076867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3e2c4-0a1d-4ede-8b12-775ee9b2e60b service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.086890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:34.102110 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be944454-d51a-483d-9ec7-9148316d44b1 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.103331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed778b1-ef27-4668-942a-71ce71c4780c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.103331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed778b1-ef27-4668-942a-71ce71c4780c admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.103743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed778b1-ef27-4668-942a-71ce71c4780c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.104532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed778b1-ef27-4668-942a-71ce71c4780c admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.104828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.109844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.109844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.110220 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed778b1-ef27-4668-942a-71ce71c4780c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.110377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] PATH_INFO: `/v3/users/136b16b3204040ed952880f41f9cd11f` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.110377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] RBAC: Authorizing `identity:delete_user(user_id=136b16b3204040ed952880f41f9cd11f)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.110377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.124146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed778b1-ef27-4668-942a-71ce71c4780c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.149046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1070/2133] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:34.149046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed778b1-ef27-4668-942a-71ce71c4780c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.172261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed778b1-ef27-4668-942a-71ce71c4780c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.181263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada44146-670c-4c96-919e-14487993bc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.182207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada44146-670c-4c96-919e-14487993bc57 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.182748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada44146-670c-4c96-919e-14487993bc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.183131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada44146-670c-4c96-919e-14487993bc57 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.184431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada44146-670c-4c96-919e-14487993bc57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.188102 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada44146-670c-4c96-919e-14487993bc57 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.289398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1071/2134] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:34 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:34.293905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada44146-670c-4c96-919e-14487993bc57 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.296468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ada44146-670c-4c96-919e-14487993bc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.372313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '136b16b3204040ed952880f41f9cd11f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.396479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada44146-670c-4c96-919e-14487993bc57 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.401136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df5584-c8cc-474b-809f-31734a143fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.401947 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df5584-c8cc-474b-809f-31734a143fec admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.402221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df5584-c8cc-474b-809f-31734a143fec admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.402493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df5584-c8cc-474b-809f-31734a143fec admin admin] PATH_INFO: `/v3/projects/e460fc409df74d5c95a427c5e962ec01/users/ac9103a951b5496c9901bf5719ec44f6/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.404436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df5584-c8cc-474b-809f-31734a143fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e460fc409df74d5c95a427c5e962ec01, user_id=ac9103a951b5496c9901bf5719ec44f6, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.407667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df5584-c8cc-474b-809f-31734a143fec admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.421345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1072/2135] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:34 2022] PUT /identity/v3/projects/e460fc409df74d5c95a427c5e962ec01/users/ac9103a951b5496c9901bf5719ec44f6/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:34.422440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df5584-c8cc-474b-809f-31734a143fec admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.477424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '136b16b3204040ed952880f41f9cd11f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.494424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '136b16b3204040ed952880f41f9cd11f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.503356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '136b16b3204040ed952880f41f9cd11f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.504429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b0d76a87-a5c5-4ed8-8412-627a6c3c308b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '136b16b3204040ed952880f41f9cd11f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:34.506627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df5584-c8cc-474b-809f-31734a143fec admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.513714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16db536-9290-4e53-bf61-51c563930c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.514884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16db536-9290-4e53-bf61-51c563930c5f admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.515428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16db536-9290-4e53-bf61-51c563930c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.515968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16db536-9290-4e53-bf61-51c563930c5f admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.517246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16db536-9290-4e53-bf61-51c563930c5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.519456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1064/2136] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:34 2022] DELETE /identity/v3/users/136b16b3204040ed952880f41f9cd11f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:34.522386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa555f-5830-4043-9f1d-c00971d5d5ea None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.522386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa555f-5830-4043-9f1d-c00971d5d5ea None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.522820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa555f-5830-4043-9f1d-c00971d5d5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.527022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16db536-9290-4e53-bf61-51c563930c5f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.568311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a0aa555f-5830-4043-9f1d-c00971d5d5ea None None] MFA Rules not processed for user `ac9103a951b5496c9901bf5719ec44f6`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:34.570046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a0aa555f-5830-4043-9f1d-c00971d5d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.650729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1073/2137] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:34 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:34.660379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16db536-9290-4e53-bf61-51c563930c5f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.663778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e16db536-9290-4e53-bf61-51c563930c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.664705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:34.692728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16db536-9290-4e53-bf61-51c563930c5f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.728318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1065/2138] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:34 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:34.730029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:34.808172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c75a2-7a7b-4d92-9845-52e49fbe2307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.809184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c75a2-7a7b-4d92-9845-52e49fbe2307 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.809664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c75a2-7a7b-4d92-9845-52e49fbe2307 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.810179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c75a2-7a7b-4d92-9845-52e49fbe2307 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.811449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c75a2-7a7b-4d92-9845-52e49fbe2307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.814804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c75a2-7a7b-4d92-9845-52e49fbe2307 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.829708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1074/2139] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:34.834929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa555f-5830-4043-9f1d-c00971d5d5ea None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.845403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa555f-5830-4043-9f1d-c00971d5d5ea None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.850479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057eecf-54c9-493a-af7f-27ab52dd56cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.851084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057eecf-54c9-493a-af7f-27ab52dd56cd admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.851357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057eecf-54c9-493a-af7f-27ab52dd56cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.851633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057eecf-54c9-493a-af7f-27ab52dd56cd admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.852278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057eecf-54c9-493a-af7f-27ab52dd56cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.853926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057eecf-54c9-493a-af7f-27ab52dd56cd admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.866573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1066/2140] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:34.868842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c75a2-7a7b-4d92-9845-52e49fbe2307 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.870209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057eecf-54c9-493a-af7f-27ab52dd56cd admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:34.871723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-185c75a2-7a7b-4d92-9845-52e49fbe2307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:34.881503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057eecf-54c9-493a-af7f-27ab52dd56cd admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.890240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3121114-b8ce-493d-8db5-cb5994d82d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.890240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3121114-b8ce-493d-8db5-cb5994d82d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.890240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3121114-b8ce-493d-8db5-cb5994d82d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.890240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3121114-b8ce-493d-8db5-cb5994d82d48 admin admin] PATH_INFO: `/v3/projects/9055b732262b47c4b677a908b9c547d1/users/ce4b1f4c53844d37aba7529b236e5f6b/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.916621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3121114-b8ce-493d-8db5-cb5994d82d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9055b732262b47c4b677a908b9c547d1, user_id=ce4b1f4c53844d37aba7529b236e5f6b, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.921110 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3121114-b8ce-493d-8db5-cb5994d82d48 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:34.960894 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c75a2-7a7b-4d92-9845-52e49fbe2307 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:34.965143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e7783-b544-42ba-b494-92a439f5ea6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:34.966062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e7783-b544-42ba-b494-92a439f5ea6f service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:34.966476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e7783-b544-42ba-b494-92a439f5ea6f service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:34.966839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e7783-b544-42ba-b494-92a439f5ea6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:34.983558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e7783-b544-42ba-b494-92a439f5ea6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:34.986201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e7783-b544-42ba-b494-92a439f5ea6f service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.003916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1075/2141] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:34 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:35.091609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1067/2142] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:34 2022] PUT /identity/v3/projects/9055b732262b47c4b677a908b9c547d1/users/ce4b1f4c53844d37aba7529b236e5f6b/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:35.109380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148846d-dbe9-44cf-b56f-bf160a0d4566 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.110709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148846d-dbe9-44cf-b56f-bf160a0d4566 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.110995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148846d-dbe9-44cf-b56f-bf160a0d4566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.141352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-7148846d-dbe9-44cf-b56f-bf160a0d4566 None None] MFA Rules not processed for user `ce4b1f4c53844d37aba7529b236e5f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:35.142682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7148846d-dbe9-44cf-b56f-bf160a0d4566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.194471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3121114-b8ce-493d-8db5-cb5994d82d48 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.197927 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e3121114-b8ce-493d-8db5-cb5994d82d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.227626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1076/2143] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:35.233656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148846d-dbe9-44cf-b56f-bf160a0d4566 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.317835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3121114-b8ce-493d-8db5-cb5994d82d48 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.324754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.325717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.326250 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.326713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] PATH_INFO: `/v3/projects/f01b7dc26b154548a9b4bdc01ced68cb` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.327501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148846d-dbe9-44cf-b56f-bf160a0d4566 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.328334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f01b7dc26b154548a9b4bdc01ced68cb)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.330499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.333987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0301548-3c15-4051-a0c1-5b619794f8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.334743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0301548-3c15-4051-a0c1-5b619794f8bc admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.335100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0301548-3c15-4051-a0c1-5b619794f8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.335312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0301548-3c15-4051-a0c1-5b619794f8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.335925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0301548-3c15-4051-a0c1-5b619794f8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.337819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0301548-3c15-4051-a0c1-5b619794f8bc admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.437679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1077/2144] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:35 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:35.437679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0301548-3c15-4051-a0c1-5b619794f8bc admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.466912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:35.492413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f01b7dc26b154548a9b4bdc01ced68cb', 'request_id': 'req-09a702c9-2ee4-4d03-8d5c-45752edaf69a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:35.499214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0301548-3c15-4051-a0c1-5b619794f8bc admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.500067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f01b7dc26b154548a9b4bdc01ced68cb', 'request_id': 'req-09a702c9-2ee4-4d03-8d5c-45752edaf69a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:35.558714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc48bd7-7de5-4a02-bc18-80912698e4e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.559915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc48bd7-7de5-4a02-bc18-80912698e4e6 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.559915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc48bd7-7de5-4a02-bc18-80912698e4e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.559915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc48bd7-7de5-4a02-bc18-80912698e4e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.561684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-edc48bd7-7de5-4a02-bc18-80912698e4e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.608678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f01b7dc26b154548a9b4bdc01ced68cb', 'request_id': 'req-09a702c9-2ee4-4d03-8d5c-45752edaf69a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:35.623318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:35.628226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] The token cache is being invalidate because project f01b7dc26b154548a9b4bdc01ced68cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:35.628226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:35.630003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:35.631772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1068/2145] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:35 2022] DELETE /identity/v3/projects/f01b7dc26b154548a9b4bdc01ced68cb => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:35.632676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.634028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.653100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc48bd7-7de5-4a02-bc18-80912698e4e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.654609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc48bd7-7de5-4a02-bc18-80912698e4e6 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.665767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1078/2146] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:35 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:35.676986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef934abf-68c1-485b-adc9-9c1e84abbbb0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.677275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef934abf-68c1-485b-adc9-9c1e84abbbb0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.677544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef934abf-68c1-485b-adc9-9c1e84abbbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.715972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ef934abf-68c1-485b-adc9-9c1e84abbbb0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:35.718247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ef934abf-68c1-485b-adc9-9c1e84abbbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.720945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a702c9-2ee4-4d03-8d5c-45752edaf69a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.727780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69f74b-1186-49c3-833f-13c952266bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.728581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69f74b-1186-49c3-833f-13c952266bab admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.728923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69f74b-1186-49c3-833f-13c952266bab admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.729252 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69f74b-1186-49c3-833f-13c952266bab admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.730190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69f74b-1186-49c3-833f-13c952266bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.732378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69f74b-1186-49c3-833f-13c952266bab admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.733936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1079/2147] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:35.738990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7d01-5e32-42ce-b302-0d5bf9667671 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.739301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7d01-5e32-42ce-b302-0d5bf9667671 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.739663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7d01-5e32-42ce-b302-0d5bf9667671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.762395 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c10b7d01-5e32-42ce-b302-0d5bf9667671 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:35.763621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c10b7d01-5e32-42ce-b302-0d5bf9667671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:35.772604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1080/2148] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:35.778335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b7d01-5e32-42ce-b302-0d5bf9667671 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.788565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b7d01-5e32-42ce-b302-0d5bf9667671 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.795183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2190be-36a7-4c39-ae7e-a36fe7200c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.795790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2190be-36a7-4c39-ae7e-a36fe7200c4f admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.796117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2190be-36a7-4c39-ae7e-a36fe7200c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.796339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2190be-36a7-4c39-ae7e-a36fe7200c4f admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.796996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2190be-36a7-4c39-ae7e-a36fe7200c4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:35.797287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2190be-36a7-4c39-ae7e-a36fe7200c4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.798899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2190be-36a7-4c39-ae7e-a36fe7200c4f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.814251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1081/2149] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:35.819616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2190be-36a7-4c39-ae7e-a36fe7200c4f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.832161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2190be-36a7-4c39-ae7e-a36fe7200c4f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.839808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c66fd4-ef50-4016-b7f8-3589eb2c4f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.841033 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c66fd4-ef50-4016-b7f8-3589eb2c4f1c admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.842976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c66fd4-ef50-4016-b7f8-3589eb2c4f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.842976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c66fd4-ef50-4016-b7f8-3589eb2c4f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.849496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c66fd4-ef50-4016-b7f8-3589eb2c4f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.850106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c66fd4-ef50-4016-b7f8-3589eb2c4f1c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.874438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1069/2150] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:35 2022] POST /identity/v3/users => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:35.885634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69f74b-1186-49c3-833f-13c952266bab admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.896748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69f74b-1186-49c3-833f-13c952266bab admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.902288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812129-fa6f-4b02-ae4c-df6415b82501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.902966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812129-fa6f-4b02-ae4c-df6415b82501 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.903360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812129-fa6f-4b02-ae4c-df6415b82501 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.903645 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812129-fa6f-4b02-ae4c-df6415b82501 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.904355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6812129-fa6f-4b02-ae4c-df6415b82501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.906046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6812129-fa6f-4b02-ae4c-df6415b82501 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.918625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1070/2151] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:35.922764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812129-fa6f-4b02-ae4c-df6415b82501 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:35.935253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812129-fa6f-4b02-ae4c-df6415b82501 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:35.941271 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c032dd-47d3-4cdd-939f-b570ef4da95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:35.941971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c032dd-47d3-4cdd-939f-b570ef4da95e admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:35.942247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c032dd-47d3-4cdd-939f-b570ef4da95e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:35.942510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c032dd-47d3-4cdd-939f-b570ef4da95e admin admin] PATH_INFO: `/v3/projects/25d83452199e4a4abc11b074a27c5a70/users/4ead453af1c94e25a5153d1a5c7feaf6/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:35.957878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c032dd-47d3-4cdd-939f-b570ef4da95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d83452199e4a4abc11b074a27c5a70, user_id=4ead453af1c94e25a5153d1a5c7feaf6, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:35.960732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c032dd-47d3-4cdd-939f-b570ef4da95e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:35.999268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1082/2152] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:35 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:36.003981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c66fd4-ef50-4016-b7f8-3589eb2c4f1c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.092997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1071/2153] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:35 2022] PUT /identity/v3/projects/25d83452199e4a4abc11b074a27c5a70/users/4ead453af1c94e25a5153d1a5c7feaf6/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:36.092997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c032dd-47d3-4cdd-939f-b570ef4da95e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.095347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-70c032dd-47d3-4cdd-939f-b570ef4da95e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:36.113454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:36.147140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c66fd4-ef50-4016-b7f8-3589eb2c4f1c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:36.207267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c032dd-47d3-4cdd-939f-b570ef4da95e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:36.213115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:36.213814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.215098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.215553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] PATH_INFO: `/v3/users/ca0d48265cfc4ae694449ccb76251c63` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.216989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0d48265cfc4ae694449ccb76251c63)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:36.219658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:36.264538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6d97f-961f-4ad5-8c31-93deb1590711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:36.264538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6d97f-961f-4ad5-8c31-93deb1590711 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.264538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6d97f-961f-4ad5-8c31-93deb1590711 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.264538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6d97f-961f-4ad5-8c31-93deb1590711 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.265963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6d97f-961f-4ad5-8c31-93deb1590711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:36.267000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6d97f-961f-4ad5-8c31-93deb1590711 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:36.438223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1083/2154] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:36 2022] POST /identity/v3/users => generated 287 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:36.439307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6d97f-961f-4ad5-8c31-93deb1590711 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.450826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6d97f-961f-4ad5-8c31-93deb1590711 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:36.457384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce5b6f-dc2d-425a-b821-b360b2b19f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:36.458103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce5b6f-dc2d-425a-b821-b360b2b19f62 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.458486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce5b6f-dc2d-425a-b821-b360b2b19f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.458820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce5b6f-dc2d-425a-b821-b360b2b19f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.459538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce5b6f-dc2d-425a-b821-b360b2b19f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:36.461241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce5b6f-dc2d-425a-b821-b360b2b19f62 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:36.473722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1084/2155] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:36.473722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce5b6f-dc2d-425a-b821-b360b2b19f62 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.485692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce5b6f-dc2d-425a-b821-b360b2b19f62 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:36.491180 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f24ebd-30cc-412f-b955-bd072c10256c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:36.491782 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f24ebd-30cc-412f-b955-bd072c10256c admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.492370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f24ebd-30cc-412f-b955-bd072c10256c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.492642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f24ebd-30cc-412f-b955-bd072c10256c admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.493332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f24ebd-30cc-412f-b955-bd072c10256c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:36.494945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f24ebd-30cc-412f-b955-bd072c10256c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:36.506452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1085/2156] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:36.510326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f24ebd-30cc-412f-b955-bd072c10256c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.520971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f24ebd-30cc-412f-b955-bd072c10256c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:36.525502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3446ec9-515d-4f0e-a41d-b43c93763358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:36.526179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3446ec9-515d-4f0e-a41d-b43c93763358 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.526486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3446ec9-515d-4f0e-a41d-b43c93763358 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.526891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3446ec9-515d-4f0e-a41d-b43c93763358 admin admin] PATH_INFO: `/v3/projects/25d83452199e4a4abc11b074a27c5a70/users/4ead453af1c94e25a5153d1a5c7feaf6/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.528782 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3446ec9-515d-4f0e-a41d-b43c93763358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d83452199e4a4abc11b074a27c5a70, user_id=4ead453af1c94e25a5153d1a5c7feaf6, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:36.531435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3446ec9-515d-4f0e-a41d-b43c93763358 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:36.543686 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1086/2157] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:36 2022] PUT /identity/v3/projects/25d83452199e4a4abc11b074a27c5a70/users/4ead453af1c94e25a5153d1a5c7feaf6/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:36.544559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3446ec9-515d-4f0e-a41d-b43c93763358 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.602908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0d48265cfc4ae694449ccb76251c63'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:36.635185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:36.639195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3446ec9-515d-4f0e-a41d-b43c93763358 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:36.644610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fae586-21d8-4a39-9b2b-77b5ef53df8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:36.645354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fae586-21d8-4a39-9b2b-77b5ef53df8f admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.645634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fae586-21d8-4a39-9b2b-77b5ef53df8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.645945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fae586-21d8-4a39-9b2b-77b5ef53df8f admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.646622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fae586-21d8-4a39-9b2b-77b5ef53df8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:36.648195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fae586-21d8-4a39-9b2b-77b5ef53df8f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:36.666134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1087/2158] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:36.666134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419409b-d857-4a63-87cb-cef990de4892 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.666134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419409b-d857-4a63-87cb-cef990de4892 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.666134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419409b-d857-4a63-87cb-cef990de4892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.694629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0d48265cfc4ae694449ccb76251c63'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:36.697303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4419409b-d857-4a63-87cb-cef990de4892 None None] MFA Rules not processed for user `4ead453af1c94e25a5153d1a5c7feaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:36.698738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4419409b-d857-4a63-87cb-cef990de4892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:36.711196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0d48265cfc4ae694449ccb76251c63'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:36.718415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0d48265cfc4ae694449ccb76251c63'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:36.719129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0d48265cfc4ae694449ccb76251c63'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:36.727119 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1072/2159] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:36 2022] DELETE /identity/v3/users/ca0d48265cfc4ae694449ccb76251c63 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:36.728053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.729433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:36.784722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1088/2160] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:36.790292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0d570-6fb5-4cd9-94b3-d61310a9db8a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.790595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0d570-6fb5-4cd9-94b3-d61310a9db8a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.790874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0d570-6fb5-4cd9-94b3-d61310a9db8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.811977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f3e0d570-6fb5-4cd9-94b3-d61310a9db8a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:36.813320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f3e0d570-6fb5-4cd9-94b3-d61310a9db8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:36.818524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d780a50-2368-47ef-8154-b2c7ccb57b88 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:36.821586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1089/2161] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:36.823470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fd9f8-4ae1-4556-8a2a-9c8a26c60202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:36.824227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fd9f8-4ae1-4556-8a2a-9c8a26c60202 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.824507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fd9f8-4ae1-4556-8a2a-9c8a26c60202 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.824845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fd9f8-4ae1-4556-8a2a-9c8a26c60202 admin admin] PATH_INFO: `/v3/projects/acf06bfc5b40422695ea0b7ace5205f7/users/2e26db0d8af449f5839d4d63494573f1/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.832815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0d570-6fb5-4cd9-94b3-d61310a9db8a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.834153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f3e0d570-6fb5-4cd9-94b3-d61310a9db8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:36.840048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fd9f8-4ae1-4556-8a2a-9c8a26c60202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf06bfc5b40422695ea0b7ace5205f7, user_id=2e26db0d8af449f5839d4d63494573f1, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:36.842788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fd9f8-4ae1-4556-8a2a-9c8a26c60202 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:36.904230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0d570-6fb5-4cd9-94b3-d61310a9db8a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:36.908239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ca0fa-eccc-428a-834f-ae45c7fc6865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:36.908903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ca0fa-eccc-428a-834f-ae45c7fc6865 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.909177 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ca0fa-eccc-428a-834f-ae45c7fc6865 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.916766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ca0fa-eccc-428a-834f-ae45c7fc6865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:36.918986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ca0fa-eccc-428a-834f-ae45c7fc6865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:36.920393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ca0fa-eccc-428a-834f-ae45c7fc6865 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:36.929917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1090/2162] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:36.930648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ca0fa-eccc-428a-834f-ae45c7fc6865 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:36.932062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2f7ca0fa-eccc-428a-834f-ae45c7fc6865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:36.969874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1073/2163] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:36 2022] PUT /identity/v3/projects/acf06bfc5b40422695ea0b7ace5205f7/users/2e26db0d8af449f5839d4d63494573f1/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:36.988267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0644d-4f6d-4fcb-b574-85c75eb1f1d4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:36.988524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0644d-4f6d-4fcb-b574-85c75eb1f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:36.988799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0644d-4f6d-4fcb-b574-85c75eb1f1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:37.017112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ca0fa-eccc-428a-834f-ae45c7fc6865 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:37.029143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-fed0644d-4f6d-4fcb-b574-85c75eb1f1d4 None None] MFA Rules not processed for user `2e26db0d8af449f5839d4d63494573f1`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:37.031030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-fed0644d-4f6d-4fcb-b574-85c75eb1f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:37.086521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1f746-0ad1-4766-b911-db8c9fac8b14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:37.087872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1f746-0ad1-4766-b911-db8c9fac8b14 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:37.088266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1f746-0ad1-4766-b911-db8c9fac8b14 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:37.089059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1f746-0ad1-4766-b911-db8c9fac8b14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:37.115986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1074/2164] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:37.146873 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0644d-4f6d-4fcb-b574-85c75eb1f1d4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:37.156910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-fed0644d-4f6d-4fcb-b574-85c75eb1f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:37.170375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0644d-4f6d-4fcb-b574-85c75eb1f1d4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:37.175964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322985e-3db4-4125-b8bc-4252025ea1a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:37.176573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322985e-3db4-4125-b8bc-4252025ea1a1 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:37.177256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322985e-3db4-4125-b8bc-4252025ea1a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:37.177536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322985e-3db4-4125-b8bc-4252025ea1a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:37.196202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322985e-3db4-4125-b8bc-4252025ea1a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:37.196202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322985e-3db4-4125-b8bc-4252025ea1a1 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:37.199061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1f746-0ad1-4766-b911-db8c9fac8b14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:37.203012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1f746-0ad1-4766-b911-db8c9fac8b14 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:37.217900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1075/2165] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:37.229528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1091/2166] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:37.290026 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:37.440591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322985e-3db4-4125-b8bc-4252025ea1a1 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:37.449489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322985e-3db4-4125-b8bc-4252025ea1a1 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:37.457114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dad96-9da2-42d3-9421-a96c87fd541c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:37.459561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dad96-9da2-42d3-9421-a96c87fd541c service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:37.459561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dad96-9da2-42d3-9421-a96c87fd541c service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:37.459561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dad96-9da2-42d3-9421-a96c87fd541c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:37.460301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-db2dad96-9da2-42d3-9421-a96c87fd541c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:37.531204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dad96-9da2-42d3-9421-a96c87fd541c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:37.532831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dad96-9da2-42d3-9421-a96c87fd541c service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:37.543043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1076/2167] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:37.609688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1f746-0ad1-4766-b911-db8c9fac8b14 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:37.610253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d2e1f746-0ad1-4766-b911-db8c9fac8b14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:37.702197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1f746-0ad1-4766-b911-db8c9fac8b14 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:37.707510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:37.708144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:37.708464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:37.708690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] PATH_INFO: `/v3/projects/e5d243b1c0624ce58d5a0242766a7de2` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:37.709879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d243b1c0624ce58d5a0242766a7de2)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:37.711784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:37.836311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d243b1c0624ce58d5a0242766a7de2', 'request_id': 'req-77e7bd43-f149-4d66-b72b-373e6d9eed46'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:37.842771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d243b1c0624ce58d5a0242766a7de2', 'request_id': 'req-77e7bd43-f149-4d66-b72b-373e6d9eed46'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:37.987110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d243b1c0624ce58d5a0242766a7de2', 'request_id': 'req-77e7bd43-f149-4d66-b72b-373e6d9eed46'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:37.999535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] The token cache is being invalidate because project e5d243b1c0624ce58d5a0242766a7de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:37.999830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:38.000508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-77e7bd43-f149-4d66-b72b-373e6d9eed46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:38.003538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1092/2168] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:37 2022] DELETE /identity/v3/projects/e5d243b1c0624ce58d5a0242766a7de2 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:38.008556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a201-04c8-4f1c-9589-85676cea7ee3 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.008839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a201-04c8-4f1c-9589-85676cea7ee3 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.009303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a201-04c8-4f1c-9589-85676cea7ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.030828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-6190a201-04c8-4f1c-9589-85676cea7ee3 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:38.032051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6190a201-04c8-4f1c-9589-85676cea7ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:38.103505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1077/2169] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:38.112640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b4b3-2378-4307-9b93-deb869f3fe71 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.112924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b4b3-2378-4307-9b93-deb869f3fe71 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.113194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b4b3-2378-4307-9b93-deb869f3fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.133429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b8f7b4b3-2378-4307-9b93-deb869f3fe71 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:38.134684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b8f7b4b3-2378-4307-9b93-deb869f3fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:38.142676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1093/2170] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:38.147837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190a201-04c8-4f1c-9589-85676cea7ee3 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.160208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190a201-04c8-4f1c-9589-85676cea7ee3 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.166552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6ccc0-e8e3-48a4-bace-b4fc2b35e235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.169187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6ccc0-e8e3-48a4-bace-b4fc2b35e235 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.169433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6ccc0-e8e3-48a4-bace-b4fc2b35e235 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.173035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6ccc0-e8e3-48a4-bace-b4fc2b35e235 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.173035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6ccc0-e8e3-48a4-bace-b4fc2b35e235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:38.173035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6ccc0-e8e3-48a4-bace-b4fc2b35e235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.173035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6ccc0-e8e3-48a4-bace-b4fc2b35e235 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.184503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1078/2171] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:38.189312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b4b3-2378-4307-9b93-deb869f3fe71 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.200023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b4b3-2378-4307-9b93-deb869f3fe71 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.204099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd1d5-47e2-4076-b870-b0c558cb4c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.204714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd1d5-47e2-4076-b870-b0c558cb4c0b admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.205013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd1d5-47e2-4076-b870-b0c558cb4c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.205771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd1d5-47e2-4076-b870-b0c558cb4c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.206576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd1d5-47e2-4076-b870-b0c558cb4c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.208447 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd1d5-47e2-4076-b870-b0c558cb4c0b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.328137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:38.385836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1094/2172] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:38 2022] POST /identity/v3/projects => generated 359 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:38.388505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6ccc0-e8e3-48a4-bace-b4fc2b35e235 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.401537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6ccc0-e8e3-48a4-bace-b4fc2b35e235 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.466592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709a10a-c753-46b2-aba0-fcdcbb24a149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.467340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709a10a-c753-46b2-aba0-fcdcbb24a149 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.467627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709a10a-c753-46b2-aba0-fcdcbb24a149 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.467858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709a10a-c753-46b2-aba0-fcdcbb24a149 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.468483 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709a10a-c753-46b2-aba0-fcdcbb24a149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.470652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709a10a-c753-46b2-aba0-fcdcbb24a149 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.599346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1079/2173] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:38 2022] POST /identity/v3/users => generated 280 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:38.605243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd1d5-47e2-4076-b870-b0c558cb4c0b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.615836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd1d5-47e2-4076-b870-b0c558cb4c0b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.620525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e664-33f8-412b-8fe7-21345dcd73f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.621193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e664-33f8-412b-8fe7-21345dcd73f4 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.621414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e664-33f8-412b-8fe7-21345dcd73f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.621684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e664-33f8-412b-8fe7-21345dcd73f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.622368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291e664-33f8-412b-8fe7-21345dcd73f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.624336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291e664-33f8-412b-8fe7-21345dcd73f4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.636308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1095/2174] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:38.641061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709a10a-c753-46b2-aba0-fcdcbb24a149 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.658076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709a10a-c753-46b2-aba0-fcdcbb24a149 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.664179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd43b92-98bb-4cc7-8ad4-e98c986798b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.664881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd43b92-98bb-4cc7-8ad4-e98c986798b9 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.665265 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd43b92-98bb-4cc7-8ad4-e98c986798b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.665654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd43b92-98bb-4cc7-8ad4-e98c986798b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.666819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd43b92-98bb-4cc7-8ad4-e98c986798b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.669518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd43b92-98bb-4cc7-8ad4-e98c986798b9 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.689758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1080/2175] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:38.696594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e664-33f8-412b-8fe7-21345dcd73f4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.706234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd43b92-98bb-4cc7-8ad4-e98c986798b9 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.708060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4fd43b92-98bb-4cc7-8ad4-e98c986798b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:38.712391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e664-33f8-412b-8fe7-21345dcd73f4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.718474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d4af1-a090-4ab5-9d4d-c12b6f4463dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.719571 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4af1-a090-4ab5-9d4d-c12b6f4463dd admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.719854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4af1-a090-4ab5-9d4d-c12b6f4463dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.720128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4af1-a090-4ab5-9d4d-c12b6f4463dd admin admin] PATH_INFO: `/v3/projects/840c307f6e804c79a91335cc9a7d4feb/users/ece92c5c74324f9c807b50d10d6b4194/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.721333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd43b92-98bb-4cc7-8ad4-e98c986798b9 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:38.726668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:38.727298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.727574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.727847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] PATH_INFO: `/v3/users/ce4b1f4c53844d37aba7529b236e5f6b` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.729004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce4b1f4c53844d37aba7529b236e5f6b)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.731082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.741274 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d4af1-a090-4ab5-9d4d-c12b6f4463dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=840c307f6e804c79a91335cc9a7d4feb, user_id=ece92c5c74324f9c807b50d10d6b4194, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:38.744845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d4af1-a090-4ab5-9d4d-c12b6f4463dd admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:38.820328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1096/2176] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:38 2022] PUT /identity/v3/projects/840c307f6e804c79a91335cc9a7d4feb/users/ece92c5c74324f9c807b50d10d6b4194/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:38.830869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5908525-ea3f-4526-a9d2-d5b897e72a18 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:38.831177 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5908525-ea3f-4526-a9d2-d5b897e72a18 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:38.831446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5908525-ea3f-4526-a9d2-d5b897e72a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:38.860577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e5908525-ea3f-4526-a9d2-d5b897e72a18 None None] MFA Rules not processed for user `ece92c5c74324f9c807b50d10d6b4194`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:38.862483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e5908525-ea3f-4526-a9d2-d5b897e72a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:38.928513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1097/2177] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:38 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:38.966584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5908525-ea3f-4526-a9d2-d5b897e72a18 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:38.967971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e5908525-ea3f-4526-a9d2-d5b897e72a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:38.988010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce4b1f4c53844d37aba7529b236e5f6b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.031358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5908525-ea3f-4526-a9d2-d5b897e72a18 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:39.035157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03f8ef-61be-440a-89f8-92dd209fa532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:39.035755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03f8ef-61be-440a-89f8-92dd209fa532 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.036023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03f8ef-61be-440a-89f8-92dd209fa532 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.036289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03f8ef-61be-440a-89f8-92dd209fa532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.044997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03f8ef-61be-440a-89f8-92dd209fa532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:39.046495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03f8ef-61be-440a-89f8-92dd209fa532 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:39.058164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1098/2178] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:39.071891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03f8ef-61be-440a-89f8-92dd209fa532 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:39.076704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce4b1f4c53844d37aba7529b236e5f6b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.089919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce4b1f4c53844d37aba7529b236e5f6b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.097566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce4b1f4c53844d37aba7529b236e5f6b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.098702 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d4e0657a-b258-408d-b606-8d3edcc6ccef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce4b1f4c53844d37aba7529b236e5f6b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.105147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1081/2179] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:38 2022] DELETE /identity/v3/users/ce4b1f4c53844d37aba7529b236e5f6b => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:39.109485 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a945f67-5ddf-4403-bc28-b4dab37f2ebc None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.109765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a945f67-5ddf-4403-bc28-b4dab37f2ebc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.110050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a945f67-5ddf-4403-bc28-b4dab37f2ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.129651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2a945f67-5ddf-4403-bc28-b4dab37f2ebc None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:39.130890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2a945f67-5ddf-4403-bc28-b4dab37f2ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:39.139703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03f8ef-61be-440a-89f8-92dd209fa532 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:39.197512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1082/2180] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:39.198821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:39.199463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.199740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.200009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] PATH_INFO: `/v3/users/ece92c5c74324f9c807b50d10d6b4194` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.201626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece92c5c74324f9c807b50d10d6b4194)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:39.203642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:39.208291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a945f67-5ddf-4403-bc28-b4dab37f2ebc None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:39.209896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2a945f67-5ddf-4403-bc28-b4dab37f2ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:39.268127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a945f67-5ddf-4403-bc28-b4dab37f2ebc None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:39.271863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de865cfc-2eb2-44d0-a64f-dffb89f0656f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:39.272456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de865cfc-2eb2-44d0-a64f-dffb89f0656f service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.272728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de865cfc-2eb2-44d0-a64f-dffb89f0656f service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.272993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de865cfc-2eb2-44d0-a64f-dffb89f0656f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.287351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de865cfc-2eb2-44d0-a64f-dffb89f0656f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:39.287351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de865cfc-2eb2-44d0-a64f-dffb89f0656f service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:39.294796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1083/2181] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:01:39.439951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece92c5c74324f9c807b50d10d6b4194'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.459674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:39.545632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece92c5c74324f9c807b50d10d6b4194'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.554604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece92c5c74324f9c807b50d10d6b4194'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.566937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece92c5c74324f9c807b50d10d6b4194'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.567664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece92c5c74324f9c807b50d10d6b4194'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:39.574920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1099/2182] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:39 2022] DELETE /identity/v3/users/ece92c5c74324f9c807b50d10d6b4194 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:39.578831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b63704-93e4-4965-b93f-8147996a46e8 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.579498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b63704-93e4-4965-b93f-8147996a46e8 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.585971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b63704-93e4-4965-b93f-8147996a46e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.585971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:39.610368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d7b63704-93e4-4965-b93f-8147996a46e8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:39.611647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d7b63704-93e4-4965-b93f-8147996a46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:39.679981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1084/2183] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:39.690493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:39.692908 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:39.757100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3f555-c9b4-4a28-9939-398a223012f8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:39.761261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe81dd-dcf2-4095-9aa4-edffd763444d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:39.761899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe81dd-dcf2-4095-9aa4-edffd763444d service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.762178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe81dd-dcf2-4095-9aa4-edffd763444d service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.762440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe81dd-dcf2-4095-9aa4-edffd763444d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.772924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe81dd-dcf2-4095-9aa4-edffd763444d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:39.774611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe81dd-dcf2-4095-9aa4-edffd763444d service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:39.785305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1100/2184] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:39.787689 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b63704-93e4-4965-b93f-8147996a46e8 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:39.789257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d7b63704-93e4-4965-b93f-8147996a46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:39.807359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b63704-93e4-4965-b93f-8147996a46e8 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:39.813559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:39.815228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:39.815570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:39.815977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] PATH_INFO: `/v3/projects/9055b732262b47c4b677a908b9c547d1` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:39.818815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9055b732262b47c4b677a908b9c547d1)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:39.819453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:40.006762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9055b732262b47c4b677a908b9c547d1', 'request_id': 'req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.013036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9055b732262b47c4b677a908b9c547d1', 'request_id': 'req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.121332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9055b732262b47c4b677a908b9c547d1', 'request_id': 'req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.134803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] The token cache is being invalidate because project 9055b732262b47c4b677a908b9c547d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:40.135188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.135951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.137632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1085/2185] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:39 2022] DELETE /identity/v3/projects/9055b732262b47c4b677a908b9c547d1 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:40.141831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe81dd-dcf2-4095-9aa4-edffd763444d service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:40.154021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe81dd-dcf2-4095-9aa4-edffd763444d service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:40.215641 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:40.216367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:40.216590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:40.216916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] PATH_INFO: `/v3/users/4ead453af1c94e25a5153d1a5c7feaf6` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:40.218144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ead453af1c94e25a5153d1a5c7feaf6)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:40.220096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:40.283782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:40.285293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:40.300678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a8e0e-67e2-489e-ad3a-6e6d14ee932b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:40.305665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:40.306344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:40.306648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:40.310819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] PATH_INFO: `/v3/projects/840c307f6e804c79a91335cc9a7d4feb` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:40.310819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=840c307f6e804c79a91335cc9a7d4feb)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:40.310819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:40.490153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ead453af1c94e25a5153d1a5c7feaf6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.494773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840c307f6e804c79a91335cc9a7d4feb', 'request_id': 'req-0141b8f6-9c0b-4d62-9408-b40edb903ed5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.501168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840c307f6e804c79a91335cc9a7d4feb', 'request_id': 'req-0141b8f6-9c0b-4d62-9408-b40edb903ed5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.557178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ead453af1c94e25a5153d1a5c7feaf6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.566356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ead453af1c94e25a5153d1a5c7feaf6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.578346 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ead453af1c94e25a5153d1a5c7feaf6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.579116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-6b9fd482-cbcc-4d43-a312-0acfc93cc5e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ead453af1c94e25a5153d1a5c7feaf6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.585129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1101/2186] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:40 2022] DELETE /identity/v3/users/4ead453af1c94e25a5153d1a5c7feaf6 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:40.676497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840c307f6e804c79a91335cc9a7d4feb', 'request_id': 'req-0141b8f6-9c0b-4d62-9408-b40edb903ed5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.688941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] The token cache is being invalidate because project 840c307f6e804c79a91335cc9a7d4feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:40.689202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.689922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-0141b8f6-9c0b-4d62-9408-b40edb903ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:40.691534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1086/2187] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:40 2022] DELETE /identity/v3/projects/840c307f6e804c79a91335cc9a7d4feb => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:40.697648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4760d3e-5e46-4198-80b4-e800d5747ce4 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:40.697968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4760d3e-5e46-4198-80b4-e800d5747ce4 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:40.698242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4760d3e-5e46-4198-80b4-e800d5747ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:40.718128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c4760d3e-5e46-4198-80b4-e800d5747ce4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:40.719340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c4760d3e-5e46-4198-80b4-e800d5747ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:40.769145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:40.791656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1102/2188] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:40.802395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710e863-cacd-414e-bf5a-a67d4d1f3752 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:40.802766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710e863-cacd-414e-bf5a-a67d4d1f3752 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:40.803001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710e863-cacd-414e-bf5a-a67d4d1f3752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:40.847038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b710e863-cacd-414e-bf5a-a67d4d1f3752 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:40.848254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b710e863-cacd-414e-bf5a-a67d4d1f3752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:40.869461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1087/2189] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:40.877207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4760d3e-5e46-4198-80b4-e800d5747ce4 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:40.888364 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4760d3e-5e46-4198-80b4-e800d5747ce4 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:40.895811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6e888-b7b9-4ef8-9e0c-39daf89a60da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:40.896523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6e888-b7b9-4ef8-9e0c-39daf89a60da admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:40.896876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6e888-b7b9-4ef8-9e0c-39daf89a60da admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:40.897218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6e888-b7b9-4ef8-9e0c-39daf89a60da admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:40.897990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6e888-b7b9-4ef8-9e0c-39daf89a60da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:40.898357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6e888-b7b9-4ef8-9e0c-39daf89a60da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:40.900130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6e888-b7b9-4ef8-9e0c-39daf89a60da admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:40.918792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1103/2190] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:40.923223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710e863-cacd-414e-bf5a-a67d4d1f3752 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:40.930601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:40.937491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710e863-cacd-414e-bf5a-a67d4d1f3752 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:40.945168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d23d7-ddc1-4268-9857-4f5d35678126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:40.946341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d23d7-ddc1-4268-9857-4f5d35678126 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:40.946487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d23d7-ddc1-4268-9857-4f5d35678126 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:40.946807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d23d7-ddc1-4268-9857-4f5d35678126 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:40.947487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d23d7-ddc1-4268-9857-4f5d35678126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:40.949574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d23d7-ddc1-4268-9857-4f5d35678126 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.080940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1088/2191] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:40 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:41.085410 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6e888-b7b9-4ef8-9e0c-39daf89a60da admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.098254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6e888-b7b9-4ef8-9e0c-39daf89a60da admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.154427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c709c1-aa52-4695-b84a-31791b1ebeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.155031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c709c1-aa52-4695-b84a-31791b1ebeea admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.155310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c709c1-aa52-4695-b84a-31791b1ebeea admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.155579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c709c1-aa52-4695-b84a-31791b1ebeea admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.156209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c709c1-aa52-4695-b84a-31791b1ebeea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.158155 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c709c1-aa52-4695-b84a-31791b1ebeea admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.176495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d23d7-ddc1-4268-9857-4f5d35678126 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.178548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-327d23d7-ddc1-4268-9857-4f5d35678126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:41.193075 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d23d7-ddc1-4268-9857-4f5d35678126 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.198461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.199197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.199501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.199871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] PATH_INFO: `/v3/projects/25d83452199e4a4abc11b074a27c5a70` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.201569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d83452199e4a4abc11b074a27c5a70)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.203080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.299778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1104/2192] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/users => generated 291 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:41.303877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c709c1-aa52-4695-b84a-31791b1ebeea admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.313536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c709c1-aa52-4695-b84a-31791b1ebeea admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.319419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3346c0-d91c-4f36-a49c-cd08cb0adb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.320003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3346c0-d91c-4f36-a49c-cd08cb0adb7b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.320280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3346c0-d91c-4f36-a49c-cd08cb0adb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.320657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3346c0-d91c-4f36-a49c-cd08cb0adb7b admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.321306 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3346c0-d91c-4f36-a49c-cd08cb0adb7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.322928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3346c0-d91c-4f36-a49c-cd08cb0adb7b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.337828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1105/2193] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:41.338577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3346c0-d91c-4f36-a49c-cd08cb0adb7b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.350570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3346c0-d91c-4f36-a49c-cd08cb0adb7b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.356401 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d83452199e4a4abc11b074a27c5a70', 'request_id': 'req-43721de3-e8ed-4593-af59-9abb07d49902'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.364286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d83452199e4a4abc11b074a27c5a70', 'request_id': 'req-43721de3-e8ed-4593-af59-9abb07d49902'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.415062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc0e11-669c-43ef-9e50-730fcd0d564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.415667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc0e11-669c-43ef-9e50-730fcd0d564d admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.416192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc0e11-669c-43ef-9e50-730fcd0d564d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.416469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc0e11-669c-43ef-9e50-730fcd0d564d admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.417134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc0e11-669c-43ef-9e50-730fcd0d564d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.418768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc0e11-669c-43ef-9e50-730fcd0d564d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.428831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1106/2194] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:41.433234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc0e11-669c-43ef-9e50-730fcd0d564d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.445209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc0e11-669c-43ef-9e50-730fcd0d564d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.450541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392856b9-79ad-4db1-a0c0-ffd071734f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.451454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392856b9-79ad-4db1-a0c0-ffd071734f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.451865 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392856b9-79ad-4db1-a0c0-ffd071734f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.452266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392856b9-79ad-4db1-a0c0-ffd071734f37 admin admin] PATH_INFO: `/v3/projects/0f20beeb6d294b609519547b3795982c/users/d3e301eb5aec4004bb9748b4cb66aaed/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.478347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392856b9-79ad-4db1-a0c0-ffd071734f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f20beeb6d294b609519547b3795982c, user_id=d3e301eb5aec4004bb9748b4cb66aaed, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.481005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392856b9-79ad-4db1-a0c0-ffd071734f37 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.499434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d83452199e4a4abc11b074a27c5a70', 'request_id': 'req-43721de3-e8ed-4593-af59-9abb07d49902'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.499660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:41.512894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] The token cache is being invalidate because project 25d83452199e4a4abc11b074a27c5a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:41.513187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.513891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-43721de3-e8ed-4593-af59-9abb07d49902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:41.515399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1089/2195] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:41 2022] DELETE /identity/v3/projects/25d83452199e4a4abc11b074a27c5a70 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:41.523276 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fe9f9-a982-46c8-bd06-8389aa1c2ba2 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.523595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fe9f9-a982-46c8-bd06-8389aa1c2ba2 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.523871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fe9f9-a982-46c8-bd06-8389aa1c2ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.545418 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9d7fe9f9-a982-46c8-bd06-8389aa1c2ba2 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:41.547225 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9d7fe9f9-a982-46c8-bd06-8389aa1c2ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:41.558866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1090/2196] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:41.568436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd07e42-f577-473e-934f-48e34d50ab7b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.568836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd07e42-f577-473e-934f-48e34d50ab7b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.569242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd07e42-f577-473e-934f-48e34d50ab7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.595586 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1bd07e42-f577-473e-934f-48e34d50ab7b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:41.595586 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1bd07e42-f577-473e-934f-48e34d50ab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:41.598054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1107/2197] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:41 2022] PUT /identity/v3/projects/0f20beeb6d294b609519547b3795982c/users/d3e301eb5aec4004bb9748b4cb66aaed/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:41.608306 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:41.618310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39ebdf-e934-4181-b97f-8abff09732b0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.619145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39ebdf-e934-4181-b97f-8abff09732b0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.619424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39ebdf-e934-4181-b97f-8abff09732b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.653609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-fc39ebdf-e934-4181-b97f-8abff09732b0 None None] MFA Rules not processed for user `d3e301eb5aec4004bb9748b4cb66aaed`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:41.655265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fc39ebdf-e934-4181-b97f-8abff09732b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:41.678035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1091/2198] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:41.679393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd07e42-f577-473e-934f-48e34d50ab7b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.688502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd07e42-f577-473e-934f-48e34d50ab7b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.693864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32665afa-e989-4803-bcf3-ffadbfeffaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.694551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32665afa-e989-4803-bcf3-ffadbfeffaea admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.694840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32665afa-e989-4803-bcf3-ffadbfeffaea admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.695114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32665afa-e989-4803-bcf3-ffadbfeffaea admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.695770 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32665afa-e989-4803-bcf3-ffadbfeffaea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:41.696074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32665afa-e989-4803-bcf3-ffadbfeffaea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.698429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32665afa-e989-4803-bcf3-ffadbfeffaea admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.710423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1092/2199] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:41.723866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32665afa-e989-4803-bcf3-ffadbfeffaea admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.732759 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1108/2200] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:41.733931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32665afa-e989-4803-bcf3-ffadbfeffaea admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.739693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ee3fd-8460-4452-b056-15510cce4ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.740312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ee3fd-8460-4452-b056-15510cce4ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.740612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ee3fd-8460-4452-b056-15510cce4ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.740898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ee3fd-8460-4452-b056-15510cce4ea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.741514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ee3fd-8460-4452-b056-15510cce4ea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.743609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ee3fd-8460-4452-b056-15510cce4ea3 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.746000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39ebdf-e934-4181-b97f-8abff09732b0 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.747470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fc39ebdf-e934-4181-b97f-8abff09732b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:41.805416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39ebdf-e934-4181-b97f-8abff09732b0 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.809512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1069cd-cda2-4b8f-8083-ffeeae772488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.810222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1069cd-cda2-4b8f-8083-ffeeae772488 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.810498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1069cd-cda2-4b8f-8083-ffeeae772488 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.810770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1069cd-cda2-4b8f-8083-ffeeae772488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.823592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1069cd-cda2-4b8f-8083-ffeeae772488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.825165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1069cd-cda2-4b8f-8083-ffeeae772488 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:41.834437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1109/2201] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:41.863033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1093/2202] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:41.867706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1069cd-cda2-4b8f-8083-ffeeae772488 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:41.880188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1069cd-cda2-4b8f-8083-ffeeae772488 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:41.953623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45abe7e-92d6-492e-921f-6541bdf51c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:41.954256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45abe7e-92d6-492e-921f-6541bdf51c15 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:41.954531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45abe7e-92d6-492e-921f-6541bdf51c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:41.954794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45abe7e-92d6-492e-921f-6541bdf51c15 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:41.955480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45abe7e-92d6-492e-921f-6541bdf51c15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:41.957504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45abe7e-92d6-492e-921f-6541bdf51c15 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.095135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1110/2203] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:41 2022] POST /identity/v3/users => generated 284 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:42.100536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ee3fd-8460-4452-b056-15510cce4ea3 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.116328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ee3fd-8460-4452-b056-15510cce4ea3 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.122361 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af0c73-f39b-4007-a44d-a4b2450a33ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.122979 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af0c73-f39b-4007-a44d-a4b2450a33ae admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.128203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af0c73-f39b-4007-a44d-a4b2450a33ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.128203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af0c73-f39b-4007-a44d-a4b2450a33ae admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.128203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af0c73-f39b-4007-a44d-a4b2450a33ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.128203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af0c73-f39b-4007-a44d-a4b2450a33ae admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.143203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1094/2204] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:42.144561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45abe7e-92d6-492e-921f-6541bdf51c15 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.159181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45abe7e-92d6-492e-921f-6541bdf51c15 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.168753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1752d50-a428-4003-82d7-c4ffea4d55b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.169408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1752d50-a428-4003-82d7-c4ffea4d55b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.173713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1752d50-a428-4003-82d7-c4ffea4d55b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.173713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1752d50-a428-4003-82d7-c4ffea4d55b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.173713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1752d50-a428-4003-82d7-c4ffea4d55b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.173713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1752d50-a428-4003-82d7-c4ffea4d55b0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.190530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1111/2205] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:01:42.196587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af0c73-f39b-4007-a44d-a4b2450a33ae admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.207951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af0c73-f39b-4007-a44d-a4b2450a33ae admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.216363 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e98180-7632-47c8-8c90-b1e723870d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.217775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e98180-7632-47c8-8c90-b1e723870d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.218173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e98180-7632-47c8-8c90-b1e723870d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.218960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e98180-7632-47c8-8c90-b1e723870d0a admin admin] PATH_INFO: `/v3/projects/9267a6ccbfa3418a9fd812fc918d8ed8/users/215bf2e25a954f329b809c5d0e5b0616/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.239360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e98180-7632-47c8-8c90-b1e723870d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9267a6ccbfa3418a9fd812fc918d8ed8, user_id=215bf2e25a954f329b809c5d0e5b0616, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.242267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e98180-7632-47c8-8c90-b1e723870d0a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.393818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1095/2206] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:42 2022] PUT /identity/v3/projects/9267a6ccbfa3418a9fd812fc918d8ed8/users/215bf2e25a954f329b809c5d0e5b0616/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:42.412570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5b31e-dffd-4793-8369-04884455cd62 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.413099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5b31e-dffd-4793-8369-04884455cd62 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.413612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5b31e-dffd-4793-8369-04884455cd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.465236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6af5b31e-dffd-4793-8369-04884455cd62 None None] MFA Rules not processed for user `215bf2e25a954f329b809c5d0e5b0616`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:42.466488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6af5b31e-dffd-4793-8369-04884455cd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:42.539584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1112/2207] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:42.552863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e98180-7632-47c8-8c90-b1e723870d0a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:42.554448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-80e98180-7632-47c8-8c90-b1e723870d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:42.620908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e98180-7632-47c8-8c90-b1e723870d0a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:42.624774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa6023-fa9f-47bd-93e9-ebb3ed1df8b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73bfe655c6d34918b21acb8748b9156d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:42.625378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa6023-fa9f-47bd-93e9-ebb3ed1df8b2 service glance] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:42.625646 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa6023-fa9f-47bd-93e9-ebb3ed1df8b2 service glance] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:42.625952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa6023-fa9f-47bd-93e9-ebb3ed1df8b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:42.643233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa6023-fa9f-47bd-93e9-ebb3ed1df8b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:42.643233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa6023-fa9f-47bd-93e9-ebb3ed1df8b2 service glance] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:42.651131 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1096/2208] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:42.790744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:42.838195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:43.968976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:44.356657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:44.848077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5b31e-dffd-4793-8369-04884455cd62 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:44.849520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6af5b31e-dffd-4793-8369-04884455cd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:44.942256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5b31e-dffd-4793-8369-04884455cd62 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:44.949386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:44.950206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:44.950519 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:44.950888 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] PATH_INFO: `/v3/users/c481e9bf83a24ef498e251c4286e1195` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:44.952235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c481e9bf83a24ef498e251c4286e1195)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:44.954707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.567756 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c481e9bf83a24ef498e251c4286e1195'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.672540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c481e9bf83a24ef498e251c4286e1195'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.685625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c481e9bf83a24ef498e251c4286e1195'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.707234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:45.711948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c481e9bf83a24ef498e251c4286e1195'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.712714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c481e9bf83a24ef498e251c4286e1195'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:45.721564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1113/2209] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:44 2022] DELETE /identity/v3/users/c481e9bf83a24ef498e251c4286e1195 => generated 0 bytes in 874 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:45.727685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8a487-0569-49e6-a381-e76ccde0265c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.728235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8a487-0569-49e6-a381-e76ccde0265c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.728426 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8a487-0569-49e6-a381-e76ccde0265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.742554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:45.757684 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b8d8a487-0569-49e6-a381-e76ccde0265c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:45.760365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b8d8a487-0569-49e6-a381-e76ccde0265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:45.835143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1097/2210] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:45.845285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:45.847570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:45.911495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a362543-bef8-4d76-b9db-4187a4372ce8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:45.912542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e4428-7215-4f5f-9231-35114a68e8cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:45.912862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e4428-7215-4f5f-9231-35114a68e8cf service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:45.913175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e4428-7215-4f5f-9231-35114a68e8cf service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:45.913521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e4428-7215-4f5f-9231-35114a68e8cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:45.924246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e4428-7215-4f5f-9231-35114a68e8cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:45.925744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e4428-7215-4f5f-9231-35114a68e8cf service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:45.936174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1114/2211] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:46.264967 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:46.530869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8a487-0569-49e6-a381-e76ccde0265c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:46.543577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b8d8a487-0569-49e6-a381-e76ccde0265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:46.555781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8a487-0569-49e6-a381-e76ccde0265c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:46.563678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:46.565713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:46.566257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:46.566754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] PATH_INFO: `/v3/projects/afc23e61e878460f98ecff82663fa6f5` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:46.568787 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc23e61e878460f98ecff82663fa6f5)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:46.572571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:46.761663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc23e61e878460f98ecff82663fa6f5', 'request_id': 'req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.771763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc23e61e878460f98ecff82663fa6f5', 'request_id': 'req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.984705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc23e61e878460f98ecff82663fa6f5', 'request_id': 'req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.996674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] The token cache is being invalidate because project afc23e61e878460f98ecff82663fa6f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:46.997096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.997837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:46.999743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1098/2212] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:46 2022] DELETE /identity/v3/projects/afc23e61e878460f98ecff82663fa6f5 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:47.004372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e4428-7215-4f5f-9231-35114a68e8cf service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:47.016644 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e4428-7215-4f5f-9231-35114a68e8cf service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:47.053514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:47.112975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:47.113643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:47.113989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:47.114216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] PATH_INFO: `/v3/users/ac9103a951b5496c9901bf5719ec44f6` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:47.115699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac9103a951b5496c9901bf5719ec44f6)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:47.117858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:47.398008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac9103a951b5496c9901bf5719ec44f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.471646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:47.504564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac9103a951b5496c9901bf5719ec44f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.515471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac9103a951b5496c9901bf5719ec44f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.529444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac9103a951b5496c9901bf5719ec44f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.530205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9d5dbb0e-789a-4dff-8c13-a4249505c4bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac9103a951b5496c9901bf5719ec44f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:47.536921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1115/2213] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:47 2022] DELETE /identity/v3/users/ac9103a951b5496c9901bf5719ec44f6 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:48.061838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:48.066556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:48.112132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed351d5e-4e0d-437c-95eb-2b9ff687cabd admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:48.184933 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:48.185565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.185874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.186148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] PATH_INFO: `/v3/projects/e460fc409df74d5c95a427c5e962ec01` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.187293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e460fc409df74d5c95a427c5e962ec01)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:48.189809 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:48.402070 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e460fc409df74d5c95a427c5e962ec01', 'request_id': 'req-a12cf052-27a8-4038-b2c3-bfe763994d28'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.410667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e460fc409df74d5c95a427c5e962ec01', 'request_id': 'req-a12cf052-27a8-4038-b2c3-bfe763994d28'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.529420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e460fc409df74d5c95a427c5e962ec01', 'request_id': 'req-a12cf052-27a8-4038-b2c3-bfe763994d28'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.557813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] The token cache is being invalidate because project e460fc409df74d5c95a427c5e962ec01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:48.558969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.560112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a12cf052-27a8-4038-b2c3-bfe763994d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:48.563730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1099/2214] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:48 2022] DELETE /identity/v3/projects/e460fc409df74d5c95a427c5e962ec01 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:48.570323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be854b-ddcf-428d-a154-ed4f32490307 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.570880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be854b-ddcf-428d-a154-ed4f32490307 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.570880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be854b-ddcf-428d-a154-ed4f32490307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.594690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-04be854b-ddcf-428d-a154-ed4f32490307 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:48.596882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-04be854b-ddcf-428d-a154-ed4f32490307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:48.607985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:48.680585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1116/2215] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:48.691856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a2292-8704-44fa-998e-de599eddc741 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.692216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a2292-8704-44fa-998e-de599eddc741 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.693604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a2292-8704-44fa-998e-de599eddc741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.713432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-231a2292-8704-44fa-998e-de599eddc741 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:48.714788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-231a2292-8704-44fa-998e-de599eddc741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:48.723062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1100/2216] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:48.727499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be854b-ddcf-428d-a154-ed4f32490307 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:48.737670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be854b-ddcf-428d-a154-ed4f32490307 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:48.742350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dee198-6ba3-4eaf-a1f6-e5733fef57ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:48.743097 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dee198-6ba3-4eaf-a1f6-e5733fef57ba admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.743406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dee198-6ba3-4eaf-a1f6-e5733fef57ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.743674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dee198-6ba3-4eaf-a1f6-e5733fef57ba admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.744324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dee198-6ba3-4eaf-a1f6-e5733fef57ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:48.744622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dee198-6ba3-4eaf-a1f6-e5733fef57ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:48.746253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dee198-6ba3-4eaf-a1f6-e5733fef57ba admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:48.762420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1117/2217] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:48.766257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a2292-8704-44fa-998e-de599eddc741 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:48.775997 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a2292-8704-44fa-998e-de599eddc741 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:48.781115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca6910-0724-40cf-95fa-3c3fe59223e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:48.781781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca6910-0724-40cf-95fa-3c3fe59223e3 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.782181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca6910-0724-40cf-95fa-3c3fe59223e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.782512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca6910-0724-40cf-95fa-3c3fe59223e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.783268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca6910-0724-40cf-95fa-3c3fe59223e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:48.785256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca6910-0724-40cf-95fa-3c3fe59223e3 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:48.843475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:48.905150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dee198-6ba3-4eaf-a1f6-e5733fef57ba admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:48.912425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1101/2218] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:48 2022] POST /identity/v3/projects => generated 407 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:48.921401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dee198-6ba3-4eaf-a1f6-e5733fef57ba admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:48.993655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc6073-f31a-41f4-9ffe-6b44213253de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:48.994390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc6073-f31a-41f4-9ffe-6b44213253de admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:48.994827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc6073-f31a-41f4-9ffe-6b44213253de admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:48.995304 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc6073-f31a-41f4-9ffe-6b44213253de admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:48.996030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc6073-f31a-41f4-9ffe-6b44213253de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:48.998176 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc6073-f31a-41f4-9ffe-6b44213253de admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:49.125049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1118/2219] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:48 2022] POST /identity/v3/users => generated 304 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:49.126471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca6910-0724-40cf-95fa-3c3fe59223e3 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:49.138672 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca6910-0724-40cf-95fa-3c3fe59223e3 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:49.143837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a0298-39c3-4c7c-a15f-60e425b71481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:49.144527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a0298-39c3-4c7c-a15f-60e425b71481 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.144875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a0298-39c3-4c7c-a15f-60e425b71481 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.145229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a0298-39c3-4c7c-a15f-60e425b71481 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.146114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a0298-39c3-4c7c-a15f-60e425b71481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:49.147794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a0298-39c3-4c7c-a15f-60e425b71481 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:49.161547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1102/2220] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:49.164441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc6073-f31a-41f4-9ffe-6b44213253de admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:49.176504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc6073-f31a-41f4-9ffe-6b44213253de admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:49.181746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a086659-c73a-455c-a7a4-9b8dfd2f819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:49.190142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a086659-c73a-455c-a7a4-9b8dfd2f819b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.190142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a086659-c73a-455c-a7a4-9b8dfd2f819b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.190142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a086659-c73a-455c-a7a4-9b8dfd2f819b admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.190142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a086659-c73a-455c-a7a4-9b8dfd2f819b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:49.190142 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a086659-c73a-455c-a7a4-9b8dfd2f819b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:49.202021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1119/2221] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:49.202220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a0298-39c3-4c7c-a15f-60e425b71481 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:49.212678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a0298-39c3-4c7c-a15f-60e425b71481 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:49.220077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75f8ee-b391-47dc-bbcd-824884fb94d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:49.221298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75f8ee-b391-47dc-bbcd-824884fb94d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.221574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75f8ee-b391-47dc-bbcd-824884fb94d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.221886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75f8ee-b391-47dc-bbcd-824884fb94d8 admin admin] PATH_INFO: `/v3/projects/31e27d2d4fff4e16af7275f26d46944a/users/979c44922edc4fc8b17a4fac661af897/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.238409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75f8ee-b391-47dc-bbcd-824884fb94d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e27d2d4fff4e16af7275f26d46944a, user_id=979c44922edc4fc8b17a4fac661af897, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:49.241124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75f8ee-b391-47dc-bbcd-824884fb94d8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:49.357233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1103/2222] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:49 2022] PUT /identity/v3/projects/31e27d2d4fff4e16af7275f26d46944a/users/979c44922edc4fc8b17a4fac661af897/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:49.368254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe325f6f-1150-46ae-ad32-6f7ee7022db9 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.368579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe325f6f-1150-46ae-ad32-6f7ee7022db9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.368886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe325f6f-1150-46ae-ad32-6f7ee7022db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.402043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-fe325f6f-1150-46ae-ad32-6f7ee7022db9 None None] MFA Rules not processed for user `979c44922edc4fc8b17a4fac661af897`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:49.403642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fe325f6f-1150-46ae-ad32-6f7ee7022db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:49.469055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:49.503712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1120/2223] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:49 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:49.503920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adae2d-2918-4e53-b659-def45d9e0221 None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.503920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adae2d-2918-4e53-b659-def45d9e0221 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.503920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adae2d-2918-4e53-b659-def45d9e0221 None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.503920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1104/2224] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 17:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:01:49.504835 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe325f6f-1150-46ae-ad32-6f7ee7022db9 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:49.521562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773284b-0e2f-48eb-ada9-4acb885068b8 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.521996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773284b-0e2f-48eb-ada9-4acb885068b8 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.522285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773284b-0e2f-48eb-ada9-4acb885068b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.546624 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0773284b-0e2f-48eb-ada9-4acb885068b8 None None] MFA Rules not processed for user `5853f9b60cf34380b218a5cda94f587d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:49.547808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0773284b-0e2f-48eb-ada9-4acb885068b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:49.578806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe325f6f-1150-46ae-ad32-6f7ee7022db9 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:49.585402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd80a1-b64c-4b34-b547-c324d9ed2d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:49.586093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd80a1-b64c-4b34-b547-c324d9ed2d88 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.586547 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd80a1-b64c-4b34-b547-c324d9ed2d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.587066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd80a1-b64c-4b34-b547-c324d9ed2d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.588154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd80a1-b64c-4b34-b547-c324d9ed2d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:49.591049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd80a1-b64c-4b34-b547-c324d9ed2d88 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:49.661955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1105/2225] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 17:01:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:49.686944 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773284b-0e2f-48eb-ada9-4acb885068b8 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:49.688461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0773284b-0e2f-48eb-ada9-4acb885068b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:49.702446 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773284b-0e2f-48eb-ada9-4acb885068b8 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:49.711627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55a54b-6c7b-4217-af72-ff5adaa50484 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:49.712249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55a54b-6c7b-4217-af72-ff5adaa50484 service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.712532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55a54b-6c7b-4217-af72-ff5adaa50484 service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.712973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55a54b-6c7b-4217-af72-ff5adaa50484 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.724289 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55a54b-6c7b-4217-af72-ff5adaa50484 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:49.727298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55a54b-6c7b-4217-af72-ff5adaa50484 service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:49.739518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1106/2226] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:49.755024 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55a54b-6c7b-4217-af72-ff5adaa50484 service placement] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:49.756513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5f55a54b-6c7b-4217-af72-ff5adaa50484 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:49.781855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55a54b-6c7b-4217-af72-ff5adaa50484 service placement] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:49.787041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:49.787695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.787972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.788245 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] PATH_INFO: `/v3/users/ae31b2903c8e458ca7c68082572db8f4` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.789396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae31b2903c8e458ca7c68082572db8f4)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:49.791602 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:49.803758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1121/2227] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:49 2022] POST /identity/v3/projects => generated 407 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:49.820798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd80a1-b64c-4b34-b547-c324d9ed2d88 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:49.886036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd80a1-b64c-4b34-b547-c324d9ed2d88 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:49.890801 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a49e1a-3c39-403b-bc0a-4cda3fcb1806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:49.891443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a49e1a-3c39-403b-bc0a-4cda3fcb1806 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:49.891774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a49e1a-3c39-403b-bc0a-4cda3fcb1806 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:49.891991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a49e1a-3c39-403b-bc0a-4cda3fcb1806 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:49.892630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a49e1a-3c39-403b-bc0a-4cda3fcb1806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:49.894597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a49e1a-3c39-403b-bc0a-4cda3fcb1806 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.043174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1122/2228] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:49 2022] POST /identity/v3/users => generated 310 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:50.049371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a49e1a-3c39-403b-bc0a-4cda3fcb1806 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.065943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a49e1a-3c39-403b-bc0a-4cda3fcb1806 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.071003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dc7d9-d2cc-4013-96a3-b6792e1878b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.071613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dc7d9-d2cc-4013-96a3-b6792e1878b9 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.072024 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dc7d9-d2cc-4013-96a3-b6792e1878b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.072175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dc7d9-d2cc-4013-96a3-b6792e1878b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.072918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31dc7d9-d2cc-4013-96a3-b6792e1878b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.074758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31dc7d9-d2cc-4013-96a3-b6792e1878b9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.087037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1123/2229] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:50.091867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dc7d9-d2cc-4013-96a3-b6792e1878b9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.094742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae31b2903c8e458ca7c68082572db8f4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.103204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dc7d9-d2cc-4013-96a3-b6792e1878b9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.108251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fc20f-0bbb-4e16-8cfe-cca5cfa3374e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.108987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fc20f-0bbb-4e16-8cfe-cca5cfa3374e admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.109264 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fc20f-0bbb-4e16-8cfe-cca5cfa3374e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.109533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fc20f-0bbb-4e16-8cfe-cca5cfa3374e admin admin] PATH_INFO: `/v3/projects/7261000e6dd444f28fe4ec23d6ef7f38/users/759b01fd98544ab59cb1c38b78933bd8/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.127979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fc20f-0bbb-4e16-8cfe-cca5cfa3374e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7261000e6dd444f28fe4ec23d6ef7f38, user_id=759b01fd98544ab59cb1c38b78933bd8, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.130705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fc20f-0bbb-4e16-8cfe-cca5cfa3374e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.202422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae31b2903c8e458ca7c68082572db8f4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.208945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1124/2230] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:50 2022] PUT /identity/v3/projects/7261000e6dd444f28fe4ec23d6ef7f38/users/759b01fd98544ab59cb1c38b78933bd8/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:50.213502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fc20f-0bbb-4e16-8cfe-cca5cfa3374e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.220253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:50.221459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae31b2903c8e458ca7c68082572db8f4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.228513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae31b2903c8e458ca7c68082572db8f4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.229309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae31b2903c8e458ca7c68082572db8f4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.235514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1107/2231] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:49 2022] DELETE /identity/v3/users/ae31b2903c8e458ca7c68082572db8f4 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:50.278600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fc20f-0bbb-4e16-8cfe-cca5cfa3374e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.342787 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a4596-87b4-495c-8013-31ac212dc60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.343333 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a4596-87b4-495c-8013-31ac212dc60a admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.343615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a4596-87b4-495c-8013-31ac212dc60a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.343884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a4596-87b4-495c-8013-31ac212dc60a admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.344534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a4596-87b4-495c-8013-31ac212dc60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.346913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a4596-87b4-495c-8013-31ac212dc60a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.362138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1125/2232] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:50.366289 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.367748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.381859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfedbf4-c450-4836-9584-09fe06ab9202 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.387197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90870554-0c27-4995-aa4b-a637bc3a1dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.388233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90870554-0c27-4995-aa4b-a637bc3a1dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.388378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90870554-0c27-4995-aa4b-a637bc3a1dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.388658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90870554-0c27-4995-aa4b-a637bc3a1dd5 admin admin] PATH_INFO: `/v3/projects/7261000e6dd444f28fe4ec23d6ef7f38/users/759b01fd98544ab59cb1c38b78933bd8/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.391204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90870554-0c27-4995-aa4b-a637bc3a1dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7261000e6dd444f28fe4ec23d6ef7f38, user_id=759b01fd98544ab59cb1c38b78933bd8, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.394235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90870554-0c27-4995-aa4b-a637bc3a1dd5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.405892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1108/2233] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:50 2022] PUT /identity/v3/projects/7261000e6dd444f28fe4ec23d6ef7f38/users/759b01fd98544ab59cb1c38b78933bd8/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:50.418796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357e775-9bf0-466a-9601-d02c6c6079e6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.419122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357e775-9bf0-466a-9601-d02c6c6079e6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.419631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357e775-9bf0-466a-9601-d02c6c6079e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.453653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a357e775-9bf0-466a-9601-d02c6c6079e6 None None] MFA Rules not processed for user `759b01fd98544ab59cb1c38b78933bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:50.455038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a357e775-9bf0-466a-9601-d02c6c6079e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.538714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1126/2234] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:50 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:50.554016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90870554-0c27-4995-aa4b-a637bc3a1dd5 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.555897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-90870554-0c27-4995-aa4b-a637bc3a1dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.574706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:50.588401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357e775-9bf0-466a-9601-d02c6c6079e6 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.592582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a357e775-9bf0-466a-9601-d02c6c6079e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.638615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90870554-0c27-4995-aa4b-a637bc3a1dd5 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.644803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c917d6e-d318-4bc9-be21-e68f3a41816c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.644803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c917d6e-d318-4bc9-be21-e68f3a41816c service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.644803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c917d6e-d318-4bc9-be21-e68f3a41816c service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.644803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c917d6e-d318-4bc9-be21-e68f3a41816c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.653326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c917d6e-d318-4bc9-be21-e68f3a41816c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.655116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c917d6e-d318-4bc9-be21-e68f3a41816c service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.667165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1109/2235] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:50 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:50.680551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357e775-9bf0-466a-9601-d02c6c6079e6 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.685709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.688237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.688401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.688627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] PATH_INFO: `/v3/projects/db9c728a8d56493db7bde4b0bc36f550` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.689905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db9c728a8d56493db7bde4b0bc36f550)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.691972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.740554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c917d6e-d318-4bc9-be21-e68f3a41816c service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:50.742080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8c917d6e-d318-4bc9-be21-e68f3a41816c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:50.760348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c917d6e-d318-4bc9-be21-e68f3a41816c service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:50.766747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:50.767823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.768113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.768389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] PATH_INFO: `/v3/users/d3e301eb5aec4004bb9748b4cb66aaed` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.769570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e301eb5aec4004bb9748b4cb66aaed)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:50.772326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:50.861500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db9c728a8d56493db7bde4b0bc36f550', 'request_id': 'req-314818d7-02f1-490a-af3c-9f414d81d2c2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.868634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db9c728a8d56493db7bde4b0bc36f550', 'request_id': 'req-314818d7-02f1-490a-af3c-9f414d81d2c2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.944294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db9c728a8d56493db7bde4b0bc36f550', 'request_id': 'req-314818d7-02f1-490a-af3c-9f414d81d2c2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.957573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] The token cache is being invalidate because project db9c728a8d56493db7bde4b0bc36f550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:50.957973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.958654 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-314818d7-02f1-490a-af3c-9f414d81d2c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:50.960274 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1127/2236] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:50 2022] DELETE /identity/v3/projects/db9c728a8d56493db7bde4b0bc36f550 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:50.964907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b71baf-7bec-4c03-880d-c602d8ea0090 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:50.965230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b71baf-7bec-4c03-880d-c602d8ea0090 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:50.965550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b71baf-7bec-4c03-880d-c602d8ea0090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:50.991785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-84b71baf-7bec-4c03-880d-c602d8ea0090 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:50.993029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-84b71baf-7bec-4c03-880d-c602d8ea0090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.041374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e301eb5aec4004bb9748b4cb66aaed'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.059607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1128/2237] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:51.068692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c57c4-b93e-44fb-b099-ca30bbea29cf None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.068985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c57c4-b93e-44fb-b099-ca30bbea29cf None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.069298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c57c4-b93e-44fb-b099-ca30bbea29cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.089652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a78c57c4-b93e-44fb-b099-ca30bbea29cf None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:51.090899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a78c57c4-b93e-44fb-b099-ca30bbea29cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.098981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1129/2238] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:51.103497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c57c4-b93e-44fb-b099-ca30bbea29cf None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.112766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c57c4-b93e-44fb-b099-ca30bbea29cf None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.124226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe795b-4ff4-494e-991e-3cc8f8a26a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.124226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe795b-4ff4-494e-991e-3cc8f8a26a54 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.124226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe795b-4ff4-494e-991e-3cc8f8a26a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.124226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe795b-4ff4-494e-991e-3cc8f8a26a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.124661 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe795b-4ff4-494e-991e-3cc8f8a26a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:51.124661 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe795b-4ff4-494e-991e-3cc8f8a26a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.124661 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe795b-4ff4-494e-991e-3cc8f8a26a54 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.134736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1130/2239] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:01:51.139167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe795b-4ff4-494e-991e-3cc8f8a26a54 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.150784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe795b-4ff4-494e-991e-3cc8f8a26a54 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.155710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24438b-08fc-474a-ad7b-413c71905c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.156089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24438b-08fc-474a-ad7b-413c71905c15 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.156367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24438b-08fc-474a-ad7b-413c71905c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.156637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24438b-08fc-474a-ad7b-413c71905c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.157258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24438b-08fc-474a-ad7b-413c71905c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.159295 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24438b-08fc-474a-ad7b-413c71905c15 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.184893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e301eb5aec4004bb9748b4cb66aaed'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.199115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e301eb5aec4004bb9748b4cb66aaed'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.207351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e301eb5aec4004bb9748b4cb66aaed'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.208090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d224aa36-2fbc-4f59-baae-182560da1f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e301eb5aec4004bb9748b4cb66aaed'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:51.224781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1110/2240] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:01:50 2022] DELETE /identity/v3/users/d3e301eb5aec4004bb9748b4cb66aaed => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:01:51.224781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf636f-6803-4e1b-bd83-63a5f9d4c5bf None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.224781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf636f-6803-4e1b-bd83-63a5f9d4c5bf None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.224781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf636f-6803-4e1b-bd83-63a5f9d4c5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.240339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-07cf636f-6803-4e1b-bd83-63a5f9d4c5bf None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:51.241552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-07cf636f-6803-4e1b-bd83-63a5f9d4c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.270752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1131/2241] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:51 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:01:51.270752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24438b-08fc-474a-ad7b-413c71905c15 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.272522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6e24438b-08fc-474a-ad7b-413c71905c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.310414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1111/2242] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:51.311853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf636f-6803-4e1b-bd83-63a5f9d4c5bf None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.314815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-07cf636f-6803-4e1b-bd83-63a5f9d4c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.329100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf636f-6803-4e1b-bd83-63a5f9d4c5bf None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.334040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:51.338387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24438b-08fc-474a-ad7b-413c71905c15 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.342632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.344717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.344717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.344717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.345618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.387105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e5e0-3c90-47b7-953a-7df881ea26f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.387709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e5e0-3c90-47b7-953a-7df881ea26f6 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.388021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e5e0-3c90-47b7-953a-7df881ea26f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.388982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63e5e0-3c90-47b7-953a-7df881ea26f6 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.389106 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63e5e0-3c90-47b7-953a-7df881ea26f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.391003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63e5e0-3c90-47b7-953a-7df881ea26f6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.405433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.406905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.416242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1132/2243] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:51 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:51.417145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.418541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.500132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b7d79-dfd4-4ca8-8178-2a09e686e38e service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.500132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9128a76-4e5e-4cac-a5ad-f0e1acde1bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.500132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9128a76-4e5e-4cac-a5ad-f0e1acde1bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.500132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9128a76-4e5e-4cac-a5ad-f0e1acde1bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.500466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9128a76-4e5e-4cac-a5ad-f0e1acde1bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.500466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9128a76-4e5e-4cac-a5ad-f0e1acde1bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.500466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9128a76-4e5e-4cac-a5ad-f0e1acde1bc5 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.507868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1133/2244] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:51.562954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1112/2245] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:51 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:51.568426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9128a76-4e5e-4cac-a5ad-f0e1acde1bc5 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.579418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9128a76-4e5e-4cac-a5ad-f0e1acde1bc5 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.584832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd61cf-e0f3-4b7a-9359-e034717e9ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.585558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd61cf-e0f3-4b7a-9359-e034717e9ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.585987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd61cf-e0f3-4b7a-9359-e034717e9ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.586360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd61cf-e0f3-4b7a-9359-e034717e9ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.587269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd61cf-e0f3-4b7a-9359-e034717e9ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.588930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd61cf-e0f3-4b7a-9359-e034717e9ec2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.605205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1134/2246] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:51.609058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e5e0-3c90-47b7-953a-7df881ea26f6 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.620319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63e5e0-3c90-47b7-953a-7df881ea26f6 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.624932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8838700-607f-4da4-9369-ef710f935038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.625532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8838700-607f-4da4-9369-ef710f935038 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.625832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8838700-607f-4da4-9369-ef710f935038 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.626102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8838700-607f-4da4-9369-ef710f935038 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.627289 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8838700-607f-4da4-9369-ef710f935038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.629615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8838700-607f-4da4-9369-ef710f935038 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.642814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1113/2247] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:51.646111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd61cf-e0f3-4b7a-9359-e034717e9ec2 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:51.655807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd61cf-e0f3-4b7a-9359-e034717e9ec2 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:51.660557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2b9fd-7b7b-4901-961e-b0c5e370d60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:51.661202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2b9fd-7b7b-4901-961e-b0c5e370d60b admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.661476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2b9fd-7b7b-4901-961e-b0c5e370d60b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.661743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2b9fd-7b7b-4901-961e-b0c5e370d60b admin admin] PATH_INFO: `/v3/projects/880cab17504d403d89b8cfff76632c6b/users/5a7ed5f2559a449fbe0b30bfeabd90df/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.677715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2b9fd-7b7b-4901-961e-b0c5e370d60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=880cab17504d403d89b8cfff76632c6b, user_id=5a7ed5f2559a449fbe0b30bfeabd90df, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:51.680438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2b9fd-7b7b-4901-961e-b0c5e370d60b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:51.837871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1135/2248] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:51 2022] PUT /identity/v3/projects/880cab17504d403d89b8cfff76632c6b/users/5a7ed5f2559a449fbe0b30bfeabd90df/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:51.844177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe2147-ff22-43e1-be50-d4ca2232d229 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:51.844517 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe2147-ff22-43e1-be50-d4ca2232d229 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:51.845609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe2147-ff22-43e1-be50-d4ca2232d229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:51.871955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ccfe2147-ff22-43e1-be50-d4ca2232d229 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:51.873275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ccfe2147-ff22-43e1-be50-d4ca2232d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:51.945258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1114/2249] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:51.955187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2b9fd-7b7b-4901-961e-b0c5e370d60b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:52.039020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2b9fd-7b7b-4901-961e-b0c5e370d60b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:52.044090 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6470e1-bf20-4af9-b9aa-5018363cbf79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:52.044987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6470e1-bf20-4af9-b9aa-5018363cbf79 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:52.045548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6470e1-bf20-4af9-b9aa-5018363cbf79 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:52.049741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6470e1-bf20-4af9-b9aa-5018363cbf79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:52.060992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6470e1-bf20-4af9-b9aa-5018363cbf79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:52.062761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6470e1-bf20-4af9-b9aa-5018363cbf79 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:52.096020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1136/2250] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:52.149460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe2147-ff22-43e1-be50-d4ca2232d229 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:52.150683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ccfe2147-ff22-43e1-be50-d4ca2232d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:52.166726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe2147-ff22-43e1-be50-d4ca2232d229 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:52.172269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:52.173031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:52.174038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:52.174411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] PATH_INFO: `/v3/projects/0f20beeb6d294b609519547b3795982c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:52.176127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f20beeb6d294b609519547b3795982c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:52.178313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:52.243808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:52.381391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f20beeb6d294b609519547b3795982c', 'request_id': 'req-69acf76f-a3c1-4666-be5f-376fc6d111c3'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.387816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f20beeb6d294b609519547b3795982c', 'request_id': 'req-69acf76f-a3c1-4666-be5f-376fc6d111c3'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.508641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f20beeb6d294b609519547b3795982c', 'request_id': 'req-69acf76f-a3c1-4666-be5f-376fc6d111c3'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.523058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] The token cache is being invalidate because project 0f20beeb6d294b609519547b3795982c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:01:52.523345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.524836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-69acf76f-a3c1-4666-be5f-376fc6d111c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:01:52.526859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1115/2251] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:01:52 2022] DELETE /identity/v3/projects/0f20beeb6d294b609519547b3795982c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:52.534212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e55e9a-4104-4829-8d1a-23c695e90fb5 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:52.534592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e55e9a-4104-4829-8d1a-23c695e90fb5 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:52.534932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e55e9a-4104-4829-8d1a-23c695e90fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:52.557311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-11e55e9a-4104-4829-8d1a-23c695e90fb5 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:52.559291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-11e55e9a-4104-4829-8d1a-23c695e90fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:52.677837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1137/2252] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:01:52.687181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f93b1-86ec-4dc3-af25-69cf325ab650 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:52.687765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f93b1-86ec-4dc3-af25-69cf325ab650 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:52.688159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f93b1-86ec-4dc3-af25-69cf325ab650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:52.713818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-bf7f93b1-86ec-4dc3-af25-69cf325ab650 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:52.715706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bf7f93b1-86ec-4dc3-af25-69cf325ab650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:52.726542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1116/2253] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:52.731705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e55e9a-4104-4829-8d1a-23c695e90fb5 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:52.749424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e55e9a-4104-4829-8d1a-23c695e90fb5 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:52.752265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25a0ba-3055-4940-b185-74e8915b24a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:52.752993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25a0ba-3055-4940-b185-74e8915b24a8 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:52.753365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25a0ba-3055-4940-b185-74e8915b24a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:52.753767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25a0ba-3055-4940-b185-74e8915b24a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:52.754909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25a0ba-3055-4940-b185-74e8915b24a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:01:52.755383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25a0ba-3055-4940-b185-74e8915b24a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:52.757152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25a0ba-3055-4940-b185-74e8915b24a8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:52.778277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1138/2254] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:01:52.783135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f93b1-86ec-4dc3-af25-69cf325ab650 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:52.806687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f93b1-86ec-4dc3-af25-69cf325ab650 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:52.806687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fae02-8242-4031-b19a-913e7cdea892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:52.806687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fae02-8242-4031-b19a-913e7cdea892 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:52.807598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fae02-8242-4031-b19a-913e7cdea892 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:52.808190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fae02-8242-4031-b19a-913e7cdea892 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:52.808966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fae02-8242-4031-b19a-913e7cdea892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:52.814938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fae02-8242-4031-b19a-913e7cdea892 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:52.825944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:52.968670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1117/2255] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:01:52 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:52.969406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25a0ba-3055-4940-b185-74e8915b24a8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:52.981820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25a0ba-3055-4940-b185-74e8915b24a8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:53.094589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ed2c5-74e0-47f7-97ce-5ad26555ff02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:53.095301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ed2c5-74e0-47f7-97ce-5ad26555ff02 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:53.095580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ed2c5-74e0-47f7-97ce-5ad26555ff02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:53.095871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ed2c5-74e0-47f7-97ce-5ad26555ff02 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:53.096515 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ed2c5-74e0-47f7-97ce-5ad26555ff02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:53.098572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ed2c5-74e0-47f7-97ce-5ad26555ff02 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:53.241588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1139/2256] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:01:52 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:01:53.248530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fae02-8242-4031-b19a-913e7cdea892 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:53.269148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fae02-8242-4031-b19a-913e7cdea892 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:53.283677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c69ee3-2c40-4a7f-a0dd-e019a6ce3d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:53.284345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c69ee3-2c40-4a7f-a0dd-e019a6ce3d9e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:53.284891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c69ee3-2c40-4a7f-a0dd-e019a6ce3d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:53.285222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c69ee3-2c40-4a7f-a0dd-e019a6ce3d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:53.286126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c69ee3-2c40-4a7f-a0dd-e019a6ce3d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:53.303118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c69ee3-2c40-4a7f-a0dd-e019a6ce3d9e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:53.304668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1118/2257] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:53.312861 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ed2c5-74e0-47f7-97ce-5ad26555ff02 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:53.334013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ed2c5-74e0-47f7-97ce-5ad26555ff02 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:53.340379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075512-f050-4e81-8e84-8275163d5013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:53.341143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075512-f050-4e81-8e84-8275163d5013 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:53.341487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075512-f050-4e81-8e84-8275163d5013 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:53.341863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075512-f050-4e81-8e84-8275163d5013 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:53.342658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8075512-f050-4e81-8e84-8275163d5013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:53.344743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8075512-f050-4e81-8e84-8275163d5013 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:53.365185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1140/2258] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:01:53.369744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c69ee3-2c40-4a7f-a0dd-e019a6ce3d9e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:53.384616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c69ee3-2c40-4a7f-a0dd-e019a6ce3d9e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:53.390654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34c8f5-ceaf-4839-8feb-b7b92634e112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:53.391963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34c8f5-ceaf-4839-8feb-b7b92634e112 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:53.392466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34c8f5-ceaf-4839-8feb-b7b92634e112 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:53.392948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34c8f5-ceaf-4839-8feb-b7b92634e112 admin admin] PATH_INFO: `/v3/projects/1b17d36bd8a444248693e01029cd8c9a/users/6ad81f1d16034a17a655475899bcd74d/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:53.414526 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34c8f5-ceaf-4839-8feb-b7b92634e112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b17d36bd8a444248693e01029cd8c9a, user_id=6ad81f1d16034a17a655475899bcd74d, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:53.417506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34c8f5-ceaf-4839-8feb-b7b92634e112 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:53.563750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1119/2259] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:01:53 2022] PUT /identity/v3/projects/1b17d36bd8a444248693e01029cd8c9a/users/6ad81f1d16034a17a655475899bcd74d/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:01:53.579536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06df9a9-bdb8-454a-8032-0539c358cdbd None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:53.585651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06df9a9-bdb8-454a-8032-0539c358cdbd None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:53.585651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06df9a9-bdb8-454a-8032-0539c358cdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:53.629457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e06df9a9-bdb8-454a-8032-0539c358cdbd None None] MFA Rules not processed for user `6ad81f1d16034a17a655475899bcd74d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:53.631453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e06df9a9-bdb8-454a-8032-0539c358cdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:53.724173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1141/2260] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:53.744275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34c8f5-ceaf-4839-8feb-b7b92634e112 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:53.748459 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7c34c8f5-ceaf-4839-8feb-b7b92634e112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:53.830854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34c8f5-ceaf-4839-8feb-b7b92634e112 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:53.836742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157abb41-9ef9-484c-a26a-0422dea28d93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73bfe655c6d34918b21acb8748b9156d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:53.837746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157abb41-9ef9-484c-a26a-0422dea28d93 service glance] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:53.838163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157abb41-9ef9-484c-a26a-0422dea28d93 service glance] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:53.838584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157abb41-9ef9-484c-a26a-0422dea28d93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:53.850953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157abb41-9ef9-484c-a26a-0422dea28d93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:53.853031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157abb41-9ef9-484c-a26a-0422dea28d93 service glance] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:53.863723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1120/2261] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:53.927968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:54.426227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:54.944913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:56.246117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:56.722916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:57.306680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:57.667105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:57.895225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:58.318374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:58.464640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:58.904657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:58.951742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:59.410949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:59.456124 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f03ca1-c761-4128-a537-4dba8c2b1d5a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:59.456878 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f03ca1-c761-4128-a537-4dba8c2b1d5a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:59.457359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f03ca1-c761-4128-a537-4dba8c2b1d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:59.497000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-14f03ca1-c761-4128-a537-4dba8c2b1d5a None None] MFA Rules not processed for user `5a7ed5f2559a449fbe0b30bfeabd90df`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:01:59.497000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-14f03ca1-c761-4128-a537-4dba8c2b1d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:59.573117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:01:59.592089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1142/2262] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:01:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:01:59.606180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157abb41-9ef9-484c-a26a-0422dea28d93 service glance] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:01:59.607626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-157abb41-9ef9-484c-a26a-0422dea28d93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:01:59.688962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157abb41-9ef9-484c-a26a-0422dea28d93 service glance] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:01:59.695379 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b383a7-4fdb-4b1f-9523-cd8c28f191be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:01:59.696000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b383a7-4fdb-4b1f-9523-cd8c28f191be service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:01:59.696281 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b383a7-4fdb-4b1f-9523-cd8c28f191be service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:01:59.696553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b383a7-4fdb-4b1f-9523-cd8c28f191be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:01:59.705481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b383a7-4fdb-4b1f-9523-cd8c28f191be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:01:59.707130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b383a7-4fdb-4b1f-9523-cd8c28f191be service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:01:59.742956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1121/2263] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:01:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:01:59.779722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:00.037129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:00.119140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:00.858708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:00.950779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:01.541873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:02.739405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f2a3d-dfdf-4a73-a0b3-2a7d38828df6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:02.740051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f2a3d-dfdf-4a73-a0b3-2a7d38828df6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:02.740051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f2a3d-dfdf-4a73-a0b3-2a7d38828df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:02.760150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1a9f2a3d-dfdf-4a73-a0b3-2a7d38828df6 None None] MFA Rules not processed for user `5a7ed5f2559a449fbe0b30bfeabd90df`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:02.761369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1a9f2a3d-dfdf-4a73-a0b3-2a7d38828df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:02.769258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1143/2264] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:09.232353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b383a7-4fdb-4b1f-9523-cd8c28f191be service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:09.233834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-37b383a7-4fdb-4b1f-9523-cd8c28f191be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:09.305815 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b383a7-4fdb-4b1f-9523-cd8c28f191be service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:09.310551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:09.311180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:09.311461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:09.311716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] PATH_INFO: `/v3/users/2e26db0d8af449f5839d4d63494573f1` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:09.312986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e26db0d8af449f5839d4d63494573f1)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:09.315168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:09.338013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f2a3d-dfdf-4a73-a0b3-2a7d38828df6 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:09.340585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1a9f2a3d-dfdf-4a73-a0b3-2a7d38828df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:09.349891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:09.354080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f2a3d-dfdf-4a73-a0b3-2a7d38828df6 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:09.358527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:09.359145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:09.359425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:09.359691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] PATH_INFO: `/v3/users/979c44922edc4fc8b17a4fac661af897` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:09.360791 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=979c44922edc4fc8b17a4fac661af897)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:09.362746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:09.513419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e26db0d8af449f5839d4d63494573f1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.615271 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '979c44922edc4fc8b17a4fac661af897'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.629474 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e26db0d8af449f5839d4d63494573f1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.642598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:09.645491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e26db0d8af449f5839d4d63494573f1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.652632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e26db0d8af449f5839d4d63494573f1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.653630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4a243181-7066-44d5-bd38-d617ceb45ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e26db0d8af449f5839d4d63494573f1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.660345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1122/2265] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:09 2022] DELETE /identity/v3/users/2e26db0d8af449f5839d4d63494573f1 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:09.664446 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9007b14-bf0c-438e-b873-fcc8539514b6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:09.664783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9007b14-bf0c-438e-b873-fcc8539514b6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:09.665307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9007b14-bf0c-438e-b873-fcc8539514b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:09.685737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a9007b14-bf0c-438e-b873-fcc8539514b6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:09.687054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a9007b14-bf0c-438e-b873-fcc8539514b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:09.728111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '979c44922edc4fc8b17a4fac661af897'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.738186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '979c44922edc4fc8b17a4fac661af897'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.773077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '979c44922edc4fc8b17a4fac661af897'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.773906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-b26be790-5e9b-442c-addd-ce3a7866da75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '979c44922edc4fc8b17a4fac661af897'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:09.780023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1144/2266] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:09 2022] DELETE /identity/v3/users/979c44922edc4fc8b17a4fac661af897 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:09.785396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fbd8d-aa6d-419f-b220-e788267f7e79 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:09.785672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fbd8d-aa6d-419f-b220-e788267f7e79 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:09.786490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fbd8d-aa6d-419f-b220-e788267f7e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:09.787666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1123/2267] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:02:09.800692 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9007b14-bf0c-438e-b873-fcc8539514b6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:09.802981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a9007b14-bf0c-438e-b873-fcc8539514b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:09.807115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-529fbd8d-aa6d-419f-b220-e788267f7e79 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:09.808285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-529fbd8d-aa6d-419f-b220-e788267f7e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:09.860190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9007b14-bf0c-438e-b873-fcc8539514b6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:09.872383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1145/2268] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:02:09.882247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fbd8d-aa6d-419f-b220-e788267f7e79 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:09.883640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-529fbd8d-aa6d-419f-b220-e788267f7e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:09.915401 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:09.916242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:09.916596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:09.916941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:09.918852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:09.933199 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:09.937115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:09.950468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1124/2269] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:02:09.954386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fbd8d-aa6d-419f-b220-e788267f7e79 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:09.957930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db0e25-d056-4687-8813-344d1d6d774b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:09.958543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db0e25-d056-4687-8813-344d1d6d774b service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:09.958831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db0e25-d056-4687-8813-344d1d6d774b service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:09.959086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db0e25-d056-4687-8813-344d1d6d774b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:09.968987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db0e25-d056-4687-8813-344d1d6d774b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:09.970551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db0e25-d056-4687-8813-344d1d6d774b service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:09.981496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1146/2270] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:02:10.192262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:10.193902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:10.206235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2b93a-7a43-463a-af5b-b455ddf4b419 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:10.211300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:10.211916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:10.212195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:10.212530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] PATH_INFO: `/v3/users/114587faee234dc38b709cb6e1d1de3a` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:10.213668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=114587faee234dc38b709cb6e1d1de3a)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:10.215647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:10.371935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:10.380054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114587faee234dc38b709cb6e1d1de3a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.459047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114587faee234dc38b709cb6e1d1de3a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.469420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db0e25-d056-4687-8813-344d1d6d774b service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:10.472040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-67db0e25-d056-4687-8813-344d1d6d774b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:10.485067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114587faee234dc38b709cb6e1d1de3a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.494493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db0e25-d056-4687-8813-344d1d6d774b service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:10.497556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114587faee234dc38b709cb6e1d1de3a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.498313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f518d4a1-0ce2-493b-a3fb-6ac38defc372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114587faee234dc38b709cb6e1d1de3a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.499326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:10.500134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:10.500563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:10.501003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] PATH_INFO: `/v3/projects/acf06bfc5b40422695ea0b7ace5205f7` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:10.502414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf06bfc5b40422695ea0b7ace5205f7)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:10.504598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:10.505988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1125/2271] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:10 2022] DELETE /identity/v3/users/114587faee234dc38b709cb6e1d1de3a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:10.510975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2c704-a7ab-4a03-b512-8499d5d76572 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:10.511264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2c704-a7ab-4a03-b512-8499d5d76572 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:10.511634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2c704-a7ab-4a03-b512-8499d5d76572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:10.532576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:10.539512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-8bb2c704-a7ab-4a03-b512-8499d5d76572 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:10.544905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8bb2c704-a7ab-4a03-b512-8499d5d76572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:10.609871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1126/2272] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:02:10.610828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2c704-a7ab-4a03-b512-8499d5d76572 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:10.612358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8bb2c704-a7ab-4a03-b512-8499d5d76572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:10.625755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2c704-a7ab-4a03-b512-8499d5d76572 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:10.631267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:10.631903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:10.632179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:10.632447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] PATH_INFO: `/v3/projects/31e27d2d4fff4e16af7275f26d46944a` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:10.633574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e27d2d4fff4e16af7275f26d46944a)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:10.635544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:10.704139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf06bfc5b40422695ea0b7ace5205f7', 'request_id': 'req-d029448c-d3f0-43cd-a3e3-d7e65817390f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.711431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf06bfc5b40422695ea0b7ace5205f7', 'request_id': 'req-d029448c-d3f0-43cd-a3e3-d7e65817390f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.804924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e27d2d4fff4e16af7275f26d46944a', 'request_id': 'req-09f54ed2-3cab-48cf-a275-bec1664becad'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.808337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf06bfc5b40422695ea0b7ace5205f7', 'request_id': 'req-d029448c-d3f0-43cd-a3e3-d7e65817390f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.812195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e27d2d4fff4e16af7275f26d46944a', 'request_id': 'req-09f54ed2-3cab-48cf-a275-bec1664becad'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.822360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] The token cache is being invalidate because project acf06bfc5b40422695ea0b7ace5205f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:10.822646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.823297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.824780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1147/2273] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:10 2022] DELETE /identity/v3/projects/acf06bfc5b40422695ea0b7ace5205f7 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:10.825536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:10.827138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:10.892577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e27d2d4fff4e16af7275f26d46944a', 'request_id': 'req-09f54ed2-3cab-48cf-a275-bec1664becad'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.906448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] The token cache is being invalidate because project 31e27d2d4fff4e16af7275f26d46944a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:10.907331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.908381 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-09f54ed2-3cab-48cf-a275-bec1664becad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:10.913001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1127/2274] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:10 2022] DELETE /identity/v3/projects/31e27d2d4fff4e16af7275f26d46944a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:10.913001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caaca4-7ae9-41f1-8326-8f6ebcd1284f None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:10.913001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caaca4-7ae9-41f1-8326-8f6ebcd1284f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:10.913001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caaca4-7ae9-41f1-8326-8f6ebcd1284f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:10.917806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029448c-d3f0-43cd-a3e3-d7e65817390f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:10.941076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-51caaca4-7ae9-41f1-8326-8f6ebcd1284f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:10.942862 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-51caaca4-7ae9-41f1-8326-8f6ebcd1284f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:10.992336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b1f6d-fc55-419f-ae83-74d115545997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:10.993535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b1f6d-fc55-419f-ae83-74d115545997 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:10.994047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b1f6d-fc55-419f-ae83-74d115545997 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:10.994313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b1f6d-fc55-419f-ae83-74d115545997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:10.996145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0e1b1f6d-fc55-419f-ae83-74d115545997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:11.000011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:11.042130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1128/2275] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:11.043259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caaca4-7ae9-41f1-8326-8f6ebcd1284f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:11.044667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-51caaca4-7ae9-41f1-8326-8f6ebcd1284f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:11.063010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caaca4-7ae9-41f1-8326-8f6ebcd1284f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:11.063010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:11.063010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.063010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.063440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] PATH_INFO: `/v3/users/759b01fd98544ab59cb1c38b78933bd8` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.064086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=759b01fd98544ab59cb1c38b78933bd8)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:11.066113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:11.071591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b1f6d-fc55-419f-ae83-74d115545997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:11.073067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b1f6d-fc55-419f-ae83-74d115545997 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:11.084974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1148/2276] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:11.086395 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87302b9-1bbf-4dca-a914-f43df52e8d1c None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.086620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87302b9-1bbf-4dca-a914-f43df52e8d1c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.087130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87302b9-1bbf-4dca-a914-f43df52e8d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.112816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e87302b9-1bbf-4dca-a914-f43df52e8d1c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:11.114711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e87302b9-1bbf-4dca-a914-f43df52e8d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:11.123545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1149/2277] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:11.128226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87302b9-1bbf-4dca-a914-f43df52e8d1c None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:11.133040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:11.138647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87302b9-1bbf-4dca-a914-f43df52e8d1c None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:11.143767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4da45-9cff-4255-a9f1-eaf1cca4498e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:11.144380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4da45-9cff-4255-a9f1-eaf1cca4498e admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.144656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4da45-9cff-4255-a9f1-eaf1cca4498e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.144935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4da45-9cff-4255-a9f1-eaf1cca4498e admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.145611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4da45-9cff-4255-a9f1-eaf1cca4498e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:02:11.146254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4da45-9cff-4255-a9f1-eaf1cca4498e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:11.147606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4da45-9cff-4255-a9f1-eaf1cca4498e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:11.161984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1150/2278] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:02:11.166843 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4da45-9cff-4255-a9f1-eaf1cca4498e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:11.181102 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4da45-9cff-4255-a9f1-eaf1cca4498e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:11.199134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e89be-e969-474d-b4df-59c1c41b3e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:11.199930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e89be-e969-474d-b4df-59c1c41b3e1c admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.200334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e89be-e969-474d-b4df-59c1c41b3e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.200640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e89be-e969-474d-b4df-59c1c41b3e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.201278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e89be-e969-474d-b4df-59c1c41b3e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:11.203359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e89be-e969-474d-b4df-59c1c41b3e1c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:11.302231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1151/2279] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:11 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:11.306736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e89be-e969-474d-b4df-59c1c41b3e1c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:11.308964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '759b01fd98544ab59cb1c38b78933bd8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:11.384011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '759b01fd98544ab59cb1c38b78933bd8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:11.402348 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '759b01fd98544ab59cb1c38b78933bd8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:11.403242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e89be-e969-474d-b4df-59c1c41b3e1c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:11.412273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b054e1-40e0-4f8b-88dd-bd9eec7733c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:11.412603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '759b01fd98544ab59cb1c38b78933bd8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:11.413333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '759b01fd98544ab59cb1c38b78933bd8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:11.420068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b054e1-40e0-4f8b-88dd-bd9eec7733c6 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.420068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b054e1-40e0-4f8b-88dd-bd9eec7733c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.420068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b054e1-40e0-4f8b-88dd-bd9eec7733c6 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.420068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b054e1-40e0-4f8b-88dd-bd9eec7733c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:11.420068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b054e1-40e0-4f8b-88dd-bd9eec7733c6 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:11.425592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1129/2280] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:11 2022] DELETE /identity/v3/users/759b01fd98544ab59cb1c38b78933bd8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:11.532547 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1152/2281] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:11 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:11.537443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:11.538955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:11.600195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b054e1-40e0-4f8b-88dd-bd9eec7733c6 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:11.605864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b4b054e1-40e0-4f8b-88dd-bd9eec7733c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:11.624862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b054e1-40e0-4f8b-88dd-bd9eec7733c6 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:11.646140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b3969-9877-4795-844f-54300d7588f0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:11.646489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:11.647720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.648079 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.648206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63147ac-0c34-4602-a222-523c5d58a519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:11.648535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] PATH_INFO: `/v3/projects/86fd78c53bab4d77bc3dc99e60559bf7` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.648683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63147ac-0c34-4602-a222-523c5d58a519 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.648963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63147ac-0c34-4602-a222-523c5d58a519 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.649319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63147ac-0c34-4602-a222-523c5d58a519 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.649911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63147ac-0c34-4602-a222-523c5d58a519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:11.650098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] RBAC: Authorizing `identity:delete_project(project_id=86fd78c53bab4d77bc3dc99e60559bf7)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:11.651586 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63147ac-0c34-4602-a222-523c5d58a519 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:11.652268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:11.663961 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1130/2282] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:11.668123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63147ac-0c34-4602-a222-523c5d58a519 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:11.686388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63147ac-0c34-4602-a222-523c5d58a519 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:11.693754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf917e0-ed6a-4610-9015-5b33b3bb09cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:11.694402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf917e0-ed6a-4610-9015-5b33b3bb09cd admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.694685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf917e0-ed6a-4610-9015-5b33b3bb09cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.694960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf917e0-ed6a-4610-9015-5b33b3bb09cd admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.695636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf917e0-ed6a-4610-9015-5b33b3bb09cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:11.697267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf917e0-ed6a-4610-9015-5b33b3bb09cd admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:11.708566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1131/2283] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:02:11.713137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf917e0-ed6a-4610-9015-5b33b3bb09cd admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:11.725455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf917e0-ed6a-4610-9015-5b33b3bb09cd admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:11.730079 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fa108-c7af-4aa0-9171-a2be60537000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:11.730748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fa108-c7af-4aa0-9171-a2be60537000 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.731022 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fa108-c7af-4aa0-9171-a2be60537000 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.731318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fa108-c7af-4aa0-9171-a2be60537000 admin admin] PATH_INFO: `/v3/projects/63b568870ecc4e4797919608bd9273ad/users/e941c29e3df944a08f18f5e029a28977/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.747390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fa108-c7af-4aa0-9171-a2be60537000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63b568870ecc4e4797919608bd9273ad, user_id=e941c29e3df944a08f18f5e029a28977, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:11.750881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fa108-c7af-4aa0-9171-a2be60537000 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:11.839250 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1132/2284] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:11 2022] PUT /identity/v3/projects/63b568870ecc4e4797919608bd9273ad/users/e941c29e3df944a08f18f5e029a28977/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:11.845958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a4543-0b04-4d83-8d93-8a408cfa429f None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:11.846329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a4543-0b04-4d83-8d93-8a408cfa429f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:11.846699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a4543-0b04-4d83-8d93-8a408cfa429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:11.855335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86fd78c53bab4d77bc3dc99e60559bf7', 'request_id': 'req-f743fbb3-6cda-4ee7-84d5-388101df654b'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:11.871919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-241a4543-0b04-4d83-8d93-8a408cfa429f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:11.873283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-241a4543-0b04-4d83-8d93-8a408cfa429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:11.887685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86fd78c53bab4d77bc3dc99e60559bf7', 'request_id': 'req-f743fbb3-6cda-4ee7-84d5-388101df654b'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:11.891105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:11.979560 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86fd78c53bab4d77bc3dc99e60559bf7', 'request_id': 'req-f743fbb3-6cda-4ee7-84d5-388101df654b'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:11.984006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1133/2285] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:11.996558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a4543-0b04-4d83-8d93-8a408cfa429f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.000557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-241a4543-0b04-4d83-8d93-8a408cfa429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:12.003789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] The token cache is being invalidate because project 86fd78c53bab4d77bc3dc99e60559bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:12.006558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:12.007521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f743fbb3-6cda-4ee7-84d5-388101df654b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:12.010203 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1153/2286] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:11 2022] DELETE /identity/v3/projects/86fd78c53bab4d77bc3dc99e60559bf7 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:12.021553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e30c-a3c4-406d-8d26-c0f0eb3e7ab0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.022211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e30c-a3c4-406d-8d26-c0f0eb3e7ab0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.022603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e30c-a3c4-406d-8d26-c0f0eb3e7ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.044739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1294e30c-a3c4-406d-8d26-c0f0eb3e7ab0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:12.046188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1294e30c-a3c4-406d-8d26-c0f0eb3e7ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:12.055103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1154/2287] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:12.065037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750f84-b383-4958-9854-399bfea498e2 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.065314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750f84-b383-4958-9854-399bfea498e2 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.065771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750f84-b383-4958-9854-399bfea498e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.089228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-83750f84-b383-4958-9854-399bfea498e2 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:12.090526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-83750f84-b383-4958-9854-399bfea498e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:12.100936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a4543-0b04-4d83-8d93-8a408cfa429f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.106742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ce0b5-b0e6-49fb-86ad-649c9e164ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.112577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ce0b5-b0e6-49fb-86ad-649c9e164ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.112577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ce0b5-b0e6-49fb-86ad-649c9e164ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.112577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ce0b5-b0e6-49fb-86ad-649c9e164ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.112577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b50ce0b5-b0e6-49fb-86ad-649c9e164ad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:12.115018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1155/2288] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:12.116396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83750f84-b383-4958-9854-399bfea498e2 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.118089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-83750f84-b383-4958-9854-399bfea498e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:12.127939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ce0b5-b0e6-49fb-86ad-649c9e164ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.128970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ce0b5-b0e6-49fb-86ad-649c9e164ad0 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.138500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83750f84-b383-4958-9854-399bfea498e2 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.152903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.153640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1134/2289] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:12.154294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.154726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.155167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] PATH_INFO: `/v3/projects/7261000e6dd444f28fe4ec23d6ef7f38` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.156588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7261000e6dd444f28fe4ec23d6ef7f38)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.158794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.158926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ce0b5-b0e6-49fb-86ad-649c9e164ad0 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.172866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ce0b5-b0e6-49fb-86ad-649c9e164ad0 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.183762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02c998-8411-4ed2-9a8e-5c676ed88ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.184493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02c998-8411-4ed2-9a8e-5c676ed88ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.185176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02c998-8411-4ed2-9a8e-5c676ed88ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.185587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02c998-8411-4ed2-9a8e-5c676ed88ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.187126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02c998-8411-4ed2-9a8e-5c676ed88ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:02:12.187509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02c998-8411-4ed2-9a8e-5c676ed88ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.190137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:12.190851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02c998-8411-4ed2-9a8e-5c676ed88ea5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.215980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1135/2290] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:02:12.222329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02c998-8411-4ed2-9a8e-5c676ed88ea5 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.237316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02c998-8411-4ed2-9a8e-5c676ed88ea5 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.244479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc340e-2b1c-4456-af8f-d1d5fb73915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.245247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc340e-2b1c-4456-af8f-d1d5fb73915c admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.245689 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc340e-2b1c-4456-af8f-d1d5fb73915c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.245953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc340e-2b1c-4456-af8f-d1d5fb73915c admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.246503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc340e-2b1c-4456-af8f-d1d5fb73915c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.249996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc340e-2b1c-4456-af8f-d1d5fb73915c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.339790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7261000e6dd444f28fe4ec23d6ef7f38', 'request_id': 'req-24e149c5-77d8-4409-83f1-58b3d4824e83'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:12.348355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7261000e6dd444f28fe4ec23d6ef7f38', 'request_id': 'req-24e149c5-77d8-4409-83f1-58b3d4824e83'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:12.369189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1136/2291] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:12.374320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc340e-2b1c-4456-af8f-d1d5fb73915c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.441649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7261000e6dd444f28fe4ec23d6ef7f38', 'request_id': 'req-24e149c5-77d8-4409-83f1-58b3d4824e83'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:12.452088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc340e-2b1c-4456-af8f-d1d5fb73915c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.457225 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543c5d2-7c48-417e-92cc-b76d7cdd7d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.457882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] The token cache is being invalidate because project 7261000e6dd444f28fe4ec23d6ef7f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:12.458153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543c5d2-7c48-417e-92cc-b76d7cdd7d56 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.458259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:12.458592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543c5d2-7c48-417e-92cc-b76d7cdd7d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.458857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-24e149c5-77d8-4409-83f1-58b3d4824e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:12.458973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543c5d2-7c48-417e-92cc-b76d7cdd7d56 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.459693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543c5d2-7c48-417e-92cc-b76d7cdd7d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.460480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1156/2292] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:12 2022] DELETE /identity/v3/projects/7261000e6dd444f28fe4ec23d6ef7f38 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:12.461877 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543c5d2-7c48-417e-92cc-b76d7cdd7d56 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.469106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e42f50-3ad3-4876-ab10-2d544310779a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.469387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e42f50-3ad3-4876-ab10-2d544310779a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.469487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e42f50-3ad3-4876-ab10-2d544310779a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.490787 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-32e42f50-3ad3-4876-ab10-2d544310779a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:12.491995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-32e42f50-3ad3-4876-ab10-2d544310779a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:12.503122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1157/2293] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:12.511497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab610e-8ba4-403e-b369-9393983baf5a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.511779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab610e-8ba4-403e-b369-9393983baf5a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.512077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab610e-8ba4-403e-b369-9393983baf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.533386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5fab610e-8ba4-403e-b369-9393983baf5a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:12.534981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5fab610e-8ba4-403e-b369-9393983baf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:12.544246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1158/2294] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:02:12.549136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab610e-8ba4-403e-b369-9393983baf5a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.561749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab610e-8ba4-403e-b369-9393983baf5a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.568720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249997a6-c602-452c-8542-51092cad0ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.569335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249997a6-c602-452c-8542-51092cad0ebd admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.569607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249997a6-c602-452c-8542-51092cad0ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.569907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249997a6-c602-452c-8542-51092cad0ebd admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.570585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249997a6-c602-452c-8542-51092cad0ebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:02:12.570881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249997a6-c602-452c-8542-51092cad0ebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.572632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249997a6-c602-452c-8542-51092cad0ebd admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.586962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1137/2295] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:12.597680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543c5d2-7c48-417e-92cc-b76d7cdd7d56 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.598229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1159/2296] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:02:12.602681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249997a6-c602-452c-8542-51092cad0ebd admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.606082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9543c5d2-7c48-417e-92cc-b76d7cdd7d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:12.616156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249997a6-c602-452c-8542-51092cad0ebd admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.621117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543c5d2-7c48-417e-92cc-b76d7cdd7d56 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.621905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cca77-7004-4c17-ab4d-62f03ef8f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.622896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cca77-7004-4c17-ab4d-62f03ef8f8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.623282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cca77-7004-4c17-ab4d-62f03ef8f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.624132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cca77-7004-4c17-ab4d-62f03ef8f8c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.625339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cca77-7004-4c17-ab4d-62f03ef8f8c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.626755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cca77-7004-4c17-ab4d-62f03ef8f8c0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.627840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1aa8c-573b-4c36-9ac3-7a0e1b123324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.628469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1aa8c-573b-4c36-9ac3-7a0e1b123324 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.628741 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1aa8c-573b-4c36-9ac3-7a0e1b123324 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.629016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1aa8c-573b-4c36-9ac3-7a0e1b123324 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.629728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e1aa8c-573b-4c36-9ac3-7a0e1b123324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.631930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e1aa8c-573b-4c36-9ac3-7a0e1b123324 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.644735 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:12.651580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1138/2297] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:12.655673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1aa8c-573b-4c36-9ac3-7a0e1b123324 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.667700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1aa8c-573b-4c36-9ac3-7a0e1b123324 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.674562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827b2cd-516e-44f5-a255-dd7ecaa1f684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.675196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827b2cd-516e-44f5-a255-dd7ecaa1f684 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.675466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827b2cd-516e-44f5-a255-dd7ecaa1f684 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.675727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827b2cd-516e-44f5-a255-dd7ecaa1f684 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.676389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827b2cd-516e-44f5-a255-dd7ecaa1f684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.684056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827b2cd-516e-44f5-a255-dd7ecaa1f684 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.698782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1139/2298] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:02:12.705674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827b2cd-516e-44f5-a255-dd7ecaa1f684 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.716797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827b2cd-516e-44f5-a255-dd7ecaa1f684 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.724715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74b44c-017c-4959-a851-4bb73fe69bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.729713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74b44c-017c-4959-a851-4bb73fe69bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.729713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74b44c-017c-4959-a851-4bb73fe69bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.729713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74b44c-017c-4959-a851-4bb73fe69bd1 admin admin] PATH_INFO: `/v3/projects/ba683c38d3e942bfa4eaae16b771787d/users/2432212b30954bc680ecb7d690e509ce/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.730186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1160/2299] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:12.733455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cca77-7004-4c17-ab4d-62f03ef8f8c0 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.743968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74b44c-017c-4959-a851-4bb73fe69bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba683c38d3e942bfa4eaae16b771787d, user_id=2432212b30954bc680ecb7d690e509ce, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.746827 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74b44c-017c-4959-a851-4bb73fe69bd1 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.802889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cca77-7004-4c17-ab4d-62f03ef8f8c0 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.808838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6222c5-a19d-4916-bef9-efcba120a523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:12.809750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6222c5-a19d-4916-bef9-efcba120a523 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.818582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6222c5-a19d-4916-bef9-efcba120a523 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.818582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6222c5-a19d-4916-bef9-efcba120a523 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.818582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6222c5-a19d-4916-bef9-efcba120a523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:12.818582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6222c5-a19d-4916-bef9-efcba120a523 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:12.846849 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1140/2300] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:12 2022] PUT /identity/v3/projects/ba683c38d3e942bfa4eaae16b771787d/users/2432212b30954bc680ecb7d690e509ce/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:12.858190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23922c-089f-4eb3-a763-a20daf4a55d6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:12.858667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23922c-089f-4eb3-a763-a20daf4a55d6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:12.858942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23922c-089f-4eb3-a763-a20daf4a55d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:12.895542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ea23922c-089f-4eb3-a763-a20daf4a55d6 None None] MFA Rules not processed for user `2432212b30954bc680ecb7d690e509ce`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:12.896960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ea23922c-089f-4eb3-a763-a20daf4a55d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:12.923738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1161/2301] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/users => generated 282 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:12.929116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6222c5-a19d-4916-bef9-efcba120a523 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:12.962202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6222c5-a19d-4916-bef9-efcba120a523 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:12.992405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1141/2302] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:02:12.997001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23922c-089f-4eb3-a763-a20daf4a55d6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:13.030492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:13.045201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861b26b-e495-44aa-a501-02d7e1f028dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:13.046043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861b26b-e495-44aa-a501-02d7e1f028dc admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.046414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861b26b-e495-44aa-a501-02d7e1f028dc admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.046791 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861b26b-e495-44aa-a501-02d7e1f028dc admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.047540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861b26b-e495-44aa-a501-02d7e1f028dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:13.049951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861b26b-e495-44aa-a501-02d7e1f028dc admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:13.061512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1162/2303] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:13.065753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861b26b-e495-44aa-a501-02d7e1f028dc admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:13.070790 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23922c-089f-4eb3-a763-a20daf4a55d6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:13.072734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e29b-2caf-4230-a7a1-23347b94a75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:13.073347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e29b-2caf-4230-a7a1-23347b94a75a admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.073620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e29b-2caf-4230-a7a1-23347b94a75a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.073959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e29b-2caf-4230-a7a1-23347b94a75a admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.074551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6e29b-2caf-4230-a7a1-23347b94a75a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:13.076495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6e29b-2caf-4230-a7a1-23347b94a75a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:13.080815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861b26b-e495-44aa-a501-02d7e1f028dc admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:13.088286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26811a0-5f74-471b-8c7e-85e27435c676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:13.090371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26811a0-5f74-471b-8c7e-85e27435c676 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.091324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26811a0-5f74-471b-8c7e-85e27435c676 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.092048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26811a0-5f74-471b-8c7e-85e27435c676 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.093067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26811a0-5f74-471b-8c7e-85e27435c676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:13.096078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26811a0-5f74-471b-8c7e-85e27435c676 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:13.108898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1163/2304] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:02:13.113018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26811a0-5f74-471b-8c7e-85e27435c676 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:13.131934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26811a0-5f74-471b-8c7e-85e27435c676 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:13.139156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a01c40-07dc-4f9b-a95a-cc7214db4cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:13.139842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a01c40-07dc-4f9b-a95a-cc7214db4cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.140119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a01c40-07dc-4f9b-a95a-cc7214db4cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.140396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a01c40-07dc-4f9b-a95a-cc7214db4cc2 admin admin] PATH_INFO: `/v3/projects/f906bc5a327545dcad07f610b6b53c8c/users/5021f06b373543dfa6df4a744e97a33f/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.156925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a01c40-07dc-4f9b-a95a-cc7214db4cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f906bc5a327545dcad07f610b6b53c8c, user_id=5021f06b373543dfa6df4a744e97a33f, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:13.159687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a01c40-07dc-4f9b-a95a-cc7214db4cc2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:13.189153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1142/2305] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:12 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:13.193993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e29b-2caf-4230-a7a1-23347b94a75a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:13.266907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:13.297217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1164/2306] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:13 2022] PUT /identity/v3/projects/f906bc5a327545dcad07f610b6b53c8c/users/5021f06b373543dfa6df4a744e97a33f/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:13.300313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e29b-2caf-4230-a7a1-23347b94a75a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:13.309235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e921a8-11fe-4161-b85d-ec369de7c59f None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.310553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e921a8-11fe-4161-b85d-ec369de7c59f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.310553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e921a8-11fe-4161-b85d-ec369de7c59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.354582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-47e921a8-11fe-4161-b85d-ec369de7c59f None None] MFA Rules not processed for user `5021f06b373543dfa6df4a744e97a33f`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:13.356207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-47e921a8-11fe-4161-b85d-ec369de7c59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:13.402759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3769f3-fc19-4984-8d56-6eb5a9b5a1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:13.403960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3769f3-fc19-4984-8d56-6eb5a9b5a1fe admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.404465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3769f3-fc19-4984-8d56-6eb5a9b5a1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.405398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3769f3-fc19-4984-8d56-6eb5a9b5a1fe admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.406593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3769f3-fc19-4984-8d56-6eb5a9b5a1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:13.409895 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3769f3-fc19-4984-8d56-6eb5a9b5a1fe admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:13.452079 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1165/2307] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:13.466120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e921a8-11fe-4161-b85d-ec369de7c59f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:13.471433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-47e921a8-11fe-4161-b85d-ec369de7c59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:13.511678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1143/2308] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:13 2022] POST /identity/v3/users => generated 291 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:13.519588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3769f3-fc19-4984-8d56-6eb5a9b5a1fe admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:13.530816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3769f3-fc19-4984-8d56-6eb5a9b5a1fe admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:13.538206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c5dab-4bde-404c-ad8f-c11d42bc2ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:13.541982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c5dab-4bde-404c-ad8f-c11d42bc2ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.541982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c5dab-4bde-404c-ad8f-c11d42bc2ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.541982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c5dab-4bde-404c-ad8f-c11d42bc2ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.542585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c5dab-4bde-404c-ad8f-c11d42bc2ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:13.544240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c5dab-4bde-404c-ad8f-c11d42bc2ef1 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:13.555227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1144/2309] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:13.563189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c5dab-4bde-404c-ad8f-c11d42bc2ef1 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:13.566188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e921a8-11fe-4161-b85d-ec369de7c59f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:13.573003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1766d2-3644-4ee0-a23a-e187cbbe8f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:13.574067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1766d2-3644-4ee0-a23a-e187cbbe8f6c service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.574505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1766d2-3644-4ee0-a23a-e187cbbe8f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.574621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c5dab-4bde-404c-ad8f-c11d42bc2ef1 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:13.574925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1766d2-3644-4ee0-a23a-e187cbbe8f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.579534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d985966-f36e-47fe-bc37-718c61745648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:13.580138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d985966-f36e-47fe-bc37-718c61745648 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.580406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d985966-f36e-47fe-bc37-718c61745648 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.580668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d985966-f36e-47fe-bc37-718c61745648 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.581342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d985966-f36e-47fe-bc37-718c61745648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:13.583508 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d985966-f36e-47fe-bc37-718c61745648 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:13.590421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1766d2-3644-4ee0-a23a-e187cbbe8f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:13.592405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1766d2-3644-4ee0-a23a-e187cbbe8f6c service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:13.596574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1145/2310] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:02:13.601592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d985966-f36e-47fe-bc37-718c61745648 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:13.607029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1166/2311] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:13.614396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d985966-f36e-47fe-bc37-718c61745648 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:13.619470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bc484-3fd3-4691-85f1-e3c69f1ee1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:13.622479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bc484-3fd3-4691-85f1-e3c69f1ee1f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.622479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bc484-3fd3-4691-85f1-e3c69f1ee1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.622479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bc484-3fd3-4691-85f1-e3c69f1ee1f4 admin admin] PATH_INFO: `/v3/projects/2c0c23ff7a434a7f9fc4f05c5310932a/users/81fa398d0d584bb5b0942c704917a0e3/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.636112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bc484-3fd3-4691-85f1-e3c69f1ee1f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0c23ff7a434a7f9fc4f05c5310932a, user_id=81fa398d0d584bb5b0942c704917a0e3, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:13.638818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bc484-3fd3-4691-85f1-e3c69f1ee1f4 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:13.744670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1146/2312] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:13 2022] PUT /identity/v3/projects/2c0c23ff7a434a7f9fc4f05c5310932a/users/81fa398d0d584bb5b0942c704917a0e3/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:13.758562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0be09-14e3-46cc-a4ba-5433e53e8f19 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.758869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0be09-14e3-46cc-a4ba-5433e53e8f19 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.759269 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0be09-14e3-46cc-a4ba-5433e53e8f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.802816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-39b0be09-14e3-46cc-a4ba-5433e53e8f19 None None] MFA Rules not processed for user `81fa398d0d584bb5b0942c704917a0e3`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:13.804042 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-39b0be09-14e3-46cc-a4ba-5433e53e8f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:13.890166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252cf7-487c-4df9-9595-16a8000615b9 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:13.897762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1167/2313] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:13.898475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252cf7-487c-4df9-9595-16a8000615b9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:13.898475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252cf7-487c-4df9-9595-16a8000615b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:13.901350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0be09-14e3-46cc-a4ba-5433e53e8f19 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:13.924548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a1252cf7-487c-4df9-9595-16a8000615b9 None None] MFA Rules not processed for user `e941c29e3df944a08f18f5e029a28977`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:13.926457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a1252cf7-487c-4df9-9595-16a8000615b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:13.983106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0be09-14e3-46cc-a4ba-5433e53e8f19 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:13.999239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f48d-6d8a-4a49-8e23-8726cd1e86c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:14.000267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f48d-6d8a-4a49-8e23-8726cd1e86c4 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:14.005914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f48d-6d8a-4a49-8e23-8726cd1e86c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:14.005914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f48d-6d8a-4a49-8e23-8726cd1e86c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:14.005914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854f48d-6d8a-4a49-8e23-8726cd1e86c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:14.011172 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854f48d-6d8a-4a49-8e23-8726cd1e86c4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:14.039001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1147/2314] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:14.067251 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1252cf7-487c-4df9-9595-16a8000615b9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:14.069368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a1252cf7-487c-4df9-9595-16a8000615b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:14.097633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:14.135062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1252cf7-487c-4df9-9595-16a8000615b9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:14.139438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e066521-2cdb-4141-93e9-2de95d479140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:14.140391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e066521-2cdb-4141-93e9-2de95d479140 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:14.140670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e066521-2cdb-4141-93e9-2de95d479140 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:14.140945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e066521-2cdb-4141-93e9-2de95d479140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:14.147465 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1168/2315] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:13 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:14.151417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e066521-2cdb-4141-93e9-2de95d479140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:14.152775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e066521-2cdb-4141-93e9-2de95d479140 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:14.153803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f48d-6d8a-4a49-8e23-8726cd1e86c4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:14.166582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1148/2316] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:14.177399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:14.245935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f48d-6d8a-4a49-8e23-8726cd1e86c4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:14.253387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d28801-da31-4736-93c5-fefa7989f354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:14.254042 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d28801-da31-4736-93c5-fefa7989f354 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:14.254350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d28801-da31-4736-93c5-fefa7989f354 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:14.254645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d28801-da31-4736-93c5-fefa7989f354 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:14.255291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d28801-da31-4736-93c5-fefa7989f354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:14.257251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d28801-da31-4736-93c5-fefa7989f354 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:14.363165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1169/2317] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:14 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:02:14.367659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e066521-2cdb-4141-93e9-2de95d479140 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:14.387390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e066521-2cdb-4141-93e9-2de95d479140 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:14.392497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c58a-03e8-4a3e-aef8-151ff12ed2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:14.393295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c58a-03e8-4a3e-aef8-151ff12ed2cc admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:14.394907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c58a-03e8-4a3e-aef8-151ff12ed2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:14.394907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c58a-03e8-4a3e-aef8-151ff12ed2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:14.395988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45c58a-03e8-4a3e-aef8-151ff12ed2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:14.397771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45c58a-03e8-4a3e-aef8-151ff12ed2cc admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:14.415712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1149/2318] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:14.430355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d28801-da31-4736-93c5-fefa7989f354 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:14.449472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d28801-da31-4736-93c5-fefa7989f354 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:14.460494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b7396-f511-4d92-bde1-01af9f121fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:14.469032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b7396-f511-4d92-bde1-01af9f121fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:14.469702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b7396-f511-4d92-bde1-01af9f121fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:14.470513 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b7396-f511-4d92-bde1-01af9f121fcf admin admin] PATH_INFO: `/v3/projects/0ff7521e998e4a069244cca279e519c2/users/4e3667cdb64b4d82a964e66deb635e38/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:14.527327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b7396-f511-4d92-bde1-01af9f121fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff7521e998e4a069244cca279e519c2, user_id=4e3667cdb64b4d82a964e66deb635e38, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:14.530580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b7396-f511-4d92-bde1-01af9f121fcf admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:14.661598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1170/2319] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:14 2022] PUT /identity/v3/projects/0ff7521e998e4a069244cca279e519c2/users/4e3667cdb64b4d82a964e66deb635e38/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:14.662438 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c58a-03e8-4a3e-aef8-151ff12ed2cc admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:14.675862 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c58a-03e8-4a3e-aef8-151ff12ed2cc admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:14.784303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1de7a-9669-44c0-b54c-9262a1e478ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:14.785427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1de7a-9669-44c0-b54c-9262a1e478ca admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:14.785427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1de7a-9669-44c0-b54c-9262a1e478ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:14.785427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1de7a-9669-44c0-b54c-9262a1e478ca admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:14.786686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1de7a-9669-44c0-b54c-9262a1e478ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:14.788456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1de7a-9669-44c0-b54c-9262a1e478ca admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:14.799862 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1150/2320] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:14.804996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b7396-f511-4d92-bde1-01af9f121fcf admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:14.816037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b7396-f511-4d92-bde1-01af9f121fcf admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:14.820758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038df1d-ec1e-4a35-a25a-67e02bae88e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:14.821501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038df1d-ec1e-4a35-a25a-67e02bae88e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:14.822830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038df1d-ec1e-4a35-a25a-67e02bae88e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:14.822830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038df1d-ec1e-4a35-a25a-67e02bae88e9 admin admin] PATH_INFO: `/v3/projects/0ff7521e998e4a069244cca279e519c2/users/4e3667cdb64b4d82a964e66deb635e38/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:14.824260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038df1d-ec1e-4a35-a25a-67e02bae88e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff7521e998e4a069244cca279e519c2, user_id=4e3667cdb64b4d82a964e66deb635e38, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:14.827128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038df1d-ec1e-4a35-a25a-67e02bae88e9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:14.838448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1171/2321] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:14 2022] PUT /identity/v3/projects/0ff7521e998e4a069244cca279e519c2/users/4e3667cdb64b4d82a964e66deb635e38/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:14.849536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a039-0d4f-46b9-88a8-a572f449a995 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:14.849924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a039-0d4f-46b9-88a8-a572f449a995 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:14.850205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a039-0d4f-46b9-88a8-a572f449a995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:14.883178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-8f33a039-0d4f-46b9-88a8-a572f449a995 None None] MFA Rules not processed for user `4e3667cdb64b4d82a964e66deb635e38`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:14.884402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8f33a039-0d4f-46b9-88a8-a572f449a995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:14.966639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1151/2322] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:14.986265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038df1d-ec1e-4a35-a25a-67e02bae88e9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:15.000842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:15.048738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038df1d-ec1e-4a35-a25a-67e02bae88e9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:15.053249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319803fa-c49a-41f2-b500-d2ab949c95f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:15.054023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319803fa-c49a-41f2-b500-d2ab949c95f6 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:15.054307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319803fa-c49a-41f2-b500-d2ab949c95f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:15.054579 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319803fa-c49a-41f2-b500-d2ab949c95f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:15.136003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319803fa-c49a-41f2-b500-d2ab949c95f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:15.137076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319803fa-c49a-41f2-b500-d2ab949c95f6 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:15.147480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1172/2323] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:17.173219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860181c-8c75-4db9-8a5e-289e0fcab34a None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:17.173969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860181c-8c75-4db9-8a5e-289e0fcab34a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:17.173969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860181c-8c75-4db9-8a5e-289e0fcab34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:17.196806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-4860181c-8c75-4db9-8a5e-289e0fcab34a None None] MFA Rules not processed for user `e941c29e3df944a08f18f5e029a28977`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:17.198033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4860181c-8c75-4db9-8a5e-289e0fcab34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:17.230383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:17.258542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1152/2324] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:17.570372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319803fa-c49a-41f2-b500-d2ab949c95f6 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:17.578532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319803fa-c49a-41f2-b500-d2ab949c95f6 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:17.582458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f3a37-78f7-4e76-aa83-6a64e4f07a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:17.583569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f3a37-78f7-4e76-aa83-6a64e4f07a96 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:17.583848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f3a37-78f7-4e76-aa83-6a64e4f07a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:17.584122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f3a37-78f7-4e76-aa83-6a64e4f07a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:17.644285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f3a37-78f7-4e76-aa83-6a64e4f07a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:17.646322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f3a37-78f7-4e76-aa83-6a64e4f07a96 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:17.655808 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1173/2325] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:19.451649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860181c-8c75-4db9-8a5e-289e0fcab34a None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:19.470483 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860181c-8c75-4db9-8a5e-289e0fcab34a None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:19.481899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae931a11-1604-4fa5-aede-61ed01101afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:19.482520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931a11-1604-4fa5-aede-61ed01101afc service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:19.482816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931a11-1604-4fa5-aede-61ed01101afc service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:19.483058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931a11-1604-4fa5-aede-61ed01101afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:19.500056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae931a11-1604-4fa5-aede-61ed01101afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:19.501894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae931a11-1604-4fa5-aede-61ed01101afc service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:19.515760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1153/2326] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:27.652170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=71 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:28.112479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f3a37-78f7-4e76-aa83-6a64e4f07a96 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:28.209707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f3a37-78f7-4e76-aa83-6a64e4f07a96 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:28.217473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:28.219921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:28.219921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:28.219921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] PATH_INFO: `/v3/users/2432212b30954bc680ecb7d690e509ce` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:28.220905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2432212b30954bc680ecb7d690e509ce)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:28.223871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:28.365675 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2432212b30954bc680ecb7d690e509ce'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.458069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2432212b30954bc680ecb7d690e509ce'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.471483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2432212b30954bc680ecb7d690e509ce'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.500805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2432212b30954bc680ecb7d690e509ce'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.501761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2432212b30954bc680ecb7d690e509ce'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:28.516286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1174/2327] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:28 2022] DELETE /identity/v3/users/2432212b30954bc680ecb7d690e509ce => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:28.520885 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e120bd2-b8e3-4232-8d1b-c602e5fb0355 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:28.521682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e120bd2-b8e3-4232-8d1b-c602e5fb0355 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:28.521682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e120bd2-b8e3-4232-8d1b-c602e5fb0355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:28.544882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-4e120bd2-b8e3-4232-8d1b-c602e5fb0355 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:28.546212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4e120bd2-b8e3-4232-8d1b-c602e5fb0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:28.623679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1154/2328] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:28.634680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:28.636259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:28.700598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986b890-2067-430b-91c0-9987d41b36d0 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:28.704454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3178f-d279-40b3-a7d3-2ffa9b29f69f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:28.705056 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3178f-d279-40b3-a7d3-2ffa9b29f69f service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:28.705329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3178f-d279-40b3-a7d3-2ffa9b29f69f service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:28.705592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3178f-d279-40b3-a7d3-2ffa9b29f69f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:28.715807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3178f-d279-40b3-a7d3-2ffa9b29f69f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:28.717266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3178f-d279-40b3-a7d3-2ffa9b29f69f service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:28.728066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1175/2329] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:02:29.180929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e120bd2-b8e3-4232-8d1b-c602e5fb0355 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:29.183086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4e120bd2-b8e3-4232-8d1b-c602e5fb0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:29.200932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e120bd2-b8e3-4232-8d1b-c602e5fb0355 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:29.207497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:29.208651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:29.210775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:29.210775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] PATH_INFO: `/v3/projects/ba683c38d3e942bfa4eaae16b771787d` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:29.210938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba683c38d3e942bfa4eaae16b771787d)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:29.213659 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:29.338250 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba683c38d3e942bfa4eaae16b771787d', 'request_id': 'req-037984a7-6cfc-48be-851c-773d12a3cbf3'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.347654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba683c38d3e942bfa4eaae16b771787d', 'request_id': 'req-037984a7-6cfc-48be-851c-773d12a3cbf3'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.430479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba683c38d3e942bfa4eaae16b771787d', 'request_id': 'req-037984a7-6cfc-48be-851c-773d12a3cbf3'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.442603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] The token cache is being invalidate because project ba683c38d3e942bfa4eaae16b771787d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:29.442980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.443567 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.445810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1155/2330] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:29 2022] DELETE /identity/v3/projects/ba683c38d3e942bfa4eaae16b771787d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:29.450349 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3178f-d279-40b3-a7d3-2ffa9b29f69f service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:29.461846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3178f-d279-40b3-a7d3-2ffa9b29f69f service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:29.488030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:29.532479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:29.533122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:29.533397 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:29.533668 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] PATH_INFO: `/v3/users/81fa398d0d584bb5b0942c704917a0e3` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:29.534839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81fa398d0d584bb5b0942c704917a0e3)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:29.536781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:29.752200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81fa398d0d584bb5b0942c704917a0e3'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.862747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81fa398d0d584bb5b0942c704917a0e3'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.872399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81fa398d0d584bb5b0942c704917a0e3'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.892144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81fa398d0d584bb5b0942c704917a0e3'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.894809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81fa398d0d584bb5b0942c704917a0e3'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:29.908315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1176/2331] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:29 2022] DELETE /identity/v3/users/81fa398d0d584bb5b0942c704917a0e3 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:30.773329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:30.775801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:30.801043 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037984a7-6cfc-48be-851c-773d12a3cbf3 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:30.921387 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:30.955519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:30.957985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:30.958481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:30.960720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] PATH_INFO: `/v3/projects/2c0c23ff7a434a7f9fc4f05c5310932a` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:30.964156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0c23ff7a434a7f9fc4f05c5310932a)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:30.986114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:31.145706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0c23ff7a434a7f9fc4f05c5310932a', 'request_id': 'req-f667e42e-92d8-4553-a90d-6d37bd278b96'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.157169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0c23ff7a434a7f9fc4f05c5310932a', 'request_id': 'req-f667e42e-92d8-4553-a90d-6d37bd278b96'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.224210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0c23ff7a434a7f9fc4f05c5310932a', 'request_id': 'req-f667e42e-92d8-4553-a90d-6d37bd278b96'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.237469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] The token cache is being invalidate because project 2c0c23ff7a434a7f9fc4f05c5310932a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:31.237964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.238745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.240414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1156/2332] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:30 2022] DELETE /identity/v3/projects/2c0c23ff7a434a7f9fc4f05c5310932a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:31.244703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:31.246432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:31.263439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2aa03-3c1f-45e6-9c62-7a4df3b59d22 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:31.369976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:31.373607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:31.374366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:31.374690 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] PATH_INFO: `/v3/users/4e3667cdb64b4d82a964e66deb635e38` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:31.376107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e3667cdb64b4d82a964e66deb635e38)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:31.380150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:31.638673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:31.645061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e3667cdb64b4d82a964e66deb635e38'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.723997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e3667cdb64b4d82a964e66deb635e38'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.736617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e3667cdb64b4d82a964e66deb635e38'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.763866 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e3667cdb64b4d82a964e66deb635e38'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.766361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-df7657d0-01d0-4ea7-accd-2fdb1c492efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e3667cdb64b4d82a964e66deb635e38'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:31.775394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1177/2333] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:02:31 2022] DELETE /identity/v3/users/4e3667cdb64b4d82a964e66deb635e38 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:33.040886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:33.044780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:33.059637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667e42e-92d8-4553-a90d-6d37bd278b96 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:33.174420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:33.174420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:33.174420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:33.174420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] PATH_INFO: `/v3/projects/0ff7521e998e4a069244cca279e519c2` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:33.175365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff7521e998e4a069244cca279e519c2)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:33.175365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:33.354728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff7521e998e4a069244cca279e519c2', 'request_id': 'req-e1b93a7b-a652-4f22-ae77-4b118c450cbe'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:33.363867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff7521e998e4a069244cca279e519c2', 'request_id': 'req-e1b93a7b-a652-4f22-ae77-4b118c450cbe'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:33.430863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff7521e998e4a069244cca279e519c2', 'request_id': 'req-e1b93a7b-a652-4f22-ae77-4b118c450cbe'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:33.470258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] The token cache is being invalidate because project 0ff7521e998e4a069244cca279e519c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:02:33.470804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:33.471732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e1b93a7b-a652-4f22-ae77-4b118c450cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:02:33.478741 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1157/2334] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:02:33 2022] DELETE /identity/v3/projects/0ff7521e998e4a069244cca279e519c2 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:02:33.480857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3a30a-e02b-4ec8-9801-25c6d873a868 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:33.481692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3a30a-e02b-4ec8-9801-25c6d873a868 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:33.482379 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3a30a-e02b-4ec8-9801-25c6d873a868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:33.522472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f0d3a30a-e02b-4ec8-9801-25c6d873a868 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:33.523809 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f0d3a30a-e02b-4ec8-9801-25c6d873a868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:33.629401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1178/2335] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:33.633671 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af38972-5f36-49bd-9286-69ebb7152c7b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:33.634144 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af38972-5f36-49bd-9286-69ebb7152c7b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:33.634468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af38972-5f36-49bd-9286-69ebb7152c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:33.666711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-5af38972-5f36-49bd-9286-69ebb7152c7b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:33.666711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5af38972-5f36-49bd-9286-69ebb7152c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:33.683634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1158/2336] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:33.684219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3a30a-e02b-4ec8-9801-25c6d873a868 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:33.695134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3a30a-e02b-4ec8-9801-25c6d873a868 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:33.700489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9800d9a-e86c-4ef2-8816-d512b4184676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:33.701100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9800d9a-e86c-4ef2-8816-d512b4184676 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:33.701375 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9800d9a-e86c-4ef2-8816-d512b4184676 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:33.701687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9800d9a-e86c-4ef2-8816-d512b4184676 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:33.702349 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9800d9a-e86c-4ef2-8816-d512b4184676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:02:33.702756 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9800d9a-e86c-4ef2-8816-d512b4184676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:33.704568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9800d9a-e86c-4ef2-8816-d512b4184676 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:33.720605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1179/2337] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:02:33.725281 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af38972-5f36-49bd-9286-69ebb7152c7b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:33.730686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:33.736187 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af38972-5f36-49bd-9286-69ebb7152c7b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:33.741918 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9c844-a08c-472e-bf9e-0dbd1e71abe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:33.742606 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9c844-a08c-472e-bf9e-0dbd1e71abe5 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:33.742951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9c844-a08c-472e-bf9e-0dbd1e71abe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:33.743304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9c844-a08c-472e-bf9e-0dbd1e71abe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:33.743941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9c844-a08c-472e-bf9e-0dbd1e71abe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:33.745983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9c844-a08c-472e-bf9e-0dbd1e71abe5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:33.873392 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1159/2338] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:33 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:33.877820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9800d9a-e86c-4ef2-8816-d512b4184676 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:33.889926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9800d9a-e86c-4ef2-8816-d512b4184676 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:33.955111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a086781-c5ba-4134-a748-5d8517848b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:33.956052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a086781-c5ba-4134-a748-5d8517848b5f admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:33.957935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a086781-c5ba-4134-a748-5d8517848b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:33.957935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a086781-c5ba-4134-a748-5d8517848b5f admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:33.957935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a086781-c5ba-4134-a748-5d8517848b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:33.961357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a086781-c5ba-4134-a748-5d8517848b5f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:34.066674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1180/2339] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:33 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:34.071174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9c844-a08c-472e-bf9e-0dbd1e71abe5 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:34.084350 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9c844-a08c-472e-bf9e-0dbd1e71abe5 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:34.091425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8db14-a633-49f1-b464-d5510c2bf02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:34.092676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8db14-a633-49f1-b464-d5510c2bf02a admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:34.092676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8db14-a633-49f1-b464-d5510c2bf02a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:34.093027 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8db14-a633-49f1-b464-d5510c2bf02a admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:34.093388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8db14-a633-49f1-b464-d5510c2bf02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:34.095329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8db14-a633-49f1-b464-d5510c2bf02a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:34.108896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1160/2340] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:34.120878 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a086781-c5ba-4134-a748-5d8517848b5f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:34.133479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a086781-c5ba-4134-a748-5d8517848b5f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:34.138777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698e783-bbe7-4b1c-b11f-effa8e1a26bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:34.148223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698e783-bbe7-4b1c-b11f-effa8e1a26bd admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:34.148223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698e783-bbe7-4b1c-b11f-effa8e1a26bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:34.148223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698e783-bbe7-4b1c-b11f-effa8e1a26bd admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:34.148223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698e783-bbe7-4b1c-b11f-effa8e1a26bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:34.148223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698e783-bbe7-4b1c-b11f-effa8e1a26bd admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:34.158580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1181/2341] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:34.164017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8db14-a633-49f1-b464-d5510c2bf02a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:34.182693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8db14-a633-49f1-b464-d5510c2bf02a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:34.187959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14363713-bef0-417f-9857-68af3b8ff198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:34.188637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14363713-bef0-417f-9857-68af3b8ff198 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:34.188916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14363713-bef0-417f-9857-68af3b8ff198 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:34.189189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14363713-bef0-417f-9857-68af3b8ff198 admin admin] PATH_INFO: `/v3/projects/42851a2730ad4d749d5e33d44ea607e6/users/9563998855204d3881af170730f04a34/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:34.214303 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14363713-bef0-417f-9857-68af3b8ff198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42851a2730ad4d749d5e33d44ea607e6, user_id=9563998855204d3881af170730f04a34, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:34.224861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14363713-bef0-417f-9857-68af3b8ff198 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:34.283636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1161/2342] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:34 2022] PUT /identity/v3/projects/42851a2730ad4d749d5e33d44ea607e6/users/9563998855204d3881af170730f04a34/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:34.302129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6598f9-d2e2-4d94-add4-913821f9f7b0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:34.302590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6598f9-d2e2-4d94-add4-913821f9f7b0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:34.302936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6598f9-d2e2-4d94-add4-913821f9f7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:34.328317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8c6598f9-d2e2-4d94-add4-913821f9f7b0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:34.328317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8c6598f9-d2e2-4d94-add4-913821f9f7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:34.374681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:34.438431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1182/2343] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:02:34.450913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14363713-bef0-417f-9857-68af3b8ff198 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:34.455035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-14363713-bef0-417f-9857-68af3b8ff198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:34.535986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14363713-bef0-417f-9857-68af3b8ff198 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:34.540534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c890-5f62-43b3-b266-05a26afc45c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:34.541145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c890-5f62-43b3-b266-05a26afc45c2 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:34.541423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c890-5f62-43b3-b266-05a26afc45c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:34.541689 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883c890-5f62-43b3-b266-05a26afc45c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:34.552045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c890-5f62-43b3-b266-05a26afc45c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:34.553597 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883c890-5f62-43b3-b266-05a26afc45c2 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:34.570406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1162/2344] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:41.480508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab7f1d-46bc-4725-b55d-1e2863da58a4 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.482202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab7f1d-46bc-4725-b55d-1e2863da58a4 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.482720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab7f1d-46bc-4725-b55d-1e2863da58a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.527074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e8ab7f1d-46bc-4725-b55d-1e2863da58a4 None None] MFA Rules not processed for user `9563998855204d3881af170730f04a34`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:41.528531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e8ab7f1d-46bc-4725-b55d-1e2863da58a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:41.696320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1183/2345] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:41.706116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c890-5f62-43b3-b266-05a26afc45c2 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:41.728875 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883c890-5f62-43b3-b266-05a26afc45c2 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:41.745834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439636b-2830-4f78-8709-6ac974e94996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:41.746107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439636b-2830-4f78-8709-6ac974e94996 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:41.746597 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439636b-2830-4f78-8709-6ac974e94996 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:41.747468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439636b-2830-4f78-8709-6ac974e94996 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:41.751676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439636b-2830-4f78-8709-6ac974e94996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:41.762027 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439636b-2830-4f78-8709-6ac974e94996 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:42.020194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1163/2346] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:02:41 2022] POST /identity/v3/projects => generated 373 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:42.037716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab7f1d-46bc-4725-b55d-1e2863da58a4 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:42.050467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab7f1d-46bc-4725-b55d-1e2863da58a4 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:42.142632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0078bdd-18c7-4d9a-bc48-1e36e48ec544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:42.143709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0078bdd-18c7-4d9a-bc48-1e36e48ec544 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:42.144050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0078bdd-18c7-4d9a-bc48-1e36e48ec544 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:42.144428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0078bdd-18c7-4d9a-bc48-1e36e48ec544 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:42.145099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0078bdd-18c7-4d9a-bc48-1e36e48ec544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:42.147539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0078bdd-18c7-4d9a-bc48-1e36e48ec544 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:42.414037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1184/2347] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:02:42 2022] POST /identity/v3/users => generated 293 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:02:42.416021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439636b-2830-4f78-8709-6ac974e94996 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:42.428389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439636b-2830-4f78-8709-6ac974e94996 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:42.433533 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d54902-a067-49d4-919e-1ee22db7c5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:42.434439 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d54902-a067-49d4-919e-1ee22db7c5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:42.434725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d54902-a067-49d4-919e-1ee22db7c5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:42.434992 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d54902-a067-49d4-919e-1ee22db7c5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:42.436545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d54902-a067-49d4-919e-1ee22db7c5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:42.438200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d54902-a067-49d4-919e-1ee22db7c5a0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:42.449487 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1164/2348] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:02:42.453775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0078bdd-18c7-4d9a-bc48-1e36e48ec544 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:42.465231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0078bdd-18c7-4d9a-bc48-1e36e48ec544 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:42.470060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4365997-b162-4f3b-b4b1-83712329782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:42.470832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4365997-b162-4f3b-b4b1-83712329782e admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:42.471116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4365997-b162-4f3b-b4b1-83712329782e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:42.471402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4365997-b162-4f3b-b4b1-83712329782e admin admin] PATH_INFO: `/v3/projects/7b0fb315a2a94894b87088aa0b8472c3/users/ca46d8f22e9749a39ff5f27986c762c9/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:42.488077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4365997-b162-4f3b-b4b1-83712329782e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0fb315a2a94894b87088aa0b8472c3, user_id=ca46d8f22e9749a39ff5f27986c762c9, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:42.491203 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4365997-b162-4f3b-b4b1-83712329782e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:42.571674 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1185/2349] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:42 2022] PUT /identity/v3/projects/7b0fb315a2a94894b87088aa0b8472c3/users/ca46d8f22e9749a39ff5f27986c762c9/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:42.575811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d54902-a067-49d4-919e-1ee22db7c5a0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:42.587296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d54902-a067-49d4-919e-1ee22db7c5a0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:42.674239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8564aa-369c-4558-b627-10ae3d4b66d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:42.675122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8564aa-369c-4558-b627-10ae3d4b66d9 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:42.675499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8564aa-369c-4558-b627-10ae3d4b66d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:42.675865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8564aa-369c-4558-b627-10ae3d4b66d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:42.676612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8564aa-369c-4558-b627-10ae3d4b66d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:42.678555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8564aa-369c-4558-b627-10ae3d4b66d9 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:42.690609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1165/2350] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:02:42.698162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4365997-b162-4f3b-b4b1-83712329782e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:42.709660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4365997-b162-4f3b-b4b1-83712329782e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:42.715406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c763c-6124-4a6e-9fcf-65b1baebd702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:42.716076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c763c-6124-4a6e-9fcf-65b1baebd702 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:42.716356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c763c-6124-4a6e-9fcf-65b1baebd702 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:42.716631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c763c-6124-4a6e-9fcf-65b1baebd702 admin admin] PATH_INFO: `/v3/projects/7b0fb315a2a94894b87088aa0b8472c3/users/ca46d8f22e9749a39ff5f27986c762c9/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:42.718632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c763c-6124-4a6e-9fcf-65b1baebd702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0fb315a2a94894b87088aa0b8472c3, user_id=ca46d8f22e9749a39ff5f27986c762c9, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:42.721371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c763c-6124-4a6e-9fcf-65b1baebd702 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:42.738716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1186/2351] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:02:42 2022] PUT /identity/v3/projects/7b0fb315a2a94894b87088aa0b8472c3/users/ca46d8f22e9749a39ff5f27986c762c9/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:02:50.633722 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7266b3-75d3-459f-9bd3-82348df04813 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:50.637100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7266b3-75d3-459f-9bd3-82348df04813 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:50.637100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7266b3-75d3-459f-9bd3-82348df04813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:50.664563 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:50.681121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ef7266b3-75d3-459f-9bd3-82348df04813 None None] MFA Rules not processed for user `ca46d8f22e9749a39ff5f27986c762c9`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:50.686310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ef7266b3-75d3-459f-9bd3-82348df04813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:50.969050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1166/2352] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:02:50.993415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c763c-6124-4a6e-9fcf-65b1baebd702 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:02:50.995489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a07c763c-6124-4a6e-9fcf-65b1baebd702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:51.008856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:02:51.118175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c763c-6124-4a6e-9fcf-65b1baebd702 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:02:51.124078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7b4e2-5915-45b7-849d-3d906600f146 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:02:51.124821 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7b4e2-5915-45b7-849d-3d906600f146 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:51.125162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7b4e2-5915-45b7-849d-3d906600f146 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:51.125504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7b4e2-5915-45b7-849d-3d906600f146 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:51.206676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7b4e2-5915-45b7-849d-3d906600f146 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:02:51.209593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7b4e2-5915-45b7-849d-3d906600f146 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:02:51.225980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1187/2353] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:02:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:02:54.300558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47569dfa-ffc2-41a6-b12b-f424f6bb734b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:02:54.301025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47569dfa-ffc2-41a6-b12b-f424f6bb734b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:02:54.301130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47569dfa-ffc2-41a6-b12b-f424f6bb734b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:02:54.344781 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-47569dfa-ffc2-41a6-b12b-f424f6bb734b None None] MFA Rules not processed for user `9563998855204d3881af170730f04a34`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:02:54.346736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-47569dfa-ffc2-41a6-b12b-f424f6bb734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:02:54.371040 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1167/2354] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:02:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:01.189451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7b4e2-5915-45b7-849d-3d906600f146 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:01.191212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-78c7b4e2-5915-45b7-849d-3d906600f146 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:01.281217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7b4e2-5915-45b7-849d-3d906600f146 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:01.291827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:01.292459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:01.292739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:01.293026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] PATH_INFO: `/v3/users/215bf2e25a954f329b809c5d0e5b0616` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:01.294258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=215bf2e25a954f329b809c5d0e5b0616)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:01.296196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:01.566774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215bf2e25a954f329b809c5d0e5b0616'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.568484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:01.681206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215bf2e25a954f329b809c5d0e5b0616'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.692210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215bf2e25a954f329b809c5d0e5b0616'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.707858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215bf2e25a954f329b809c5d0e5b0616'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.708724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215bf2e25a954f329b809c5d0e5b0616'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:01.715657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1188/2355] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:01 2022] DELETE /identity/v3/users/215bf2e25a954f329b809c5d0e5b0616 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:01.720171 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93248d10-69de-4199-94a1-2a1984633c7d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:01.720697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93248d10-69de-4199-94a1-2a1984633c7d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:01.725276 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93248d10-69de-4199-94a1-2a1984633c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:01.740149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-93248d10-69de-4199-94a1-2a1984633c7d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:01.741480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-93248d10-69de-4199-94a1-2a1984633c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:01.804292 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1168/2356] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:01.815023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:01.816446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:01.883756 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb343a-10f6-4acc-a972-f98a98f5d7fe admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:01.887393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c932a-7460-41e8-993d-56bee8b71475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:01.888003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c932a-7460-41e8-993d-56bee8b71475 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:01.888282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c932a-7460-41e8-993d-56bee8b71475 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:01.888551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c932a-7460-41e8-993d-56bee8b71475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:01.897817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c932a-7460-41e8-993d-56bee8b71475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:01.899353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c932a-7460-41e8-993d-56bee8b71475 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:01.909003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1189/2357] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:02.347244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93248d10-69de-4199-94a1-2a1984633c7d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:02.348717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-93248d10-69de-4199-94a1-2a1984633c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:02.361128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93248d10-69de-4199-94a1-2a1984633c7d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:02.366830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:02.373076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:02.373076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:02.373076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] PATH_INFO: `/v3/projects/9267a6ccbfa3418a9fd812fc918d8ed8` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:02.373076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9267a6ccbfa3418a9fd812fc918d8ed8)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:02.373076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:02.494324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9267a6ccbfa3418a9fd812fc918d8ed8', 'request_id': 'req-76760dac-cdce-4691-9f4c-f38c5728b68a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.500488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9267a6ccbfa3418a9fd812fc918d8ed8', 'request_id': 'req-76760dac-cdce-4691-9f4c-f38c5728b68a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.624464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9267a6ccbfa3418a9fd812fc918d8ed8', 'request_id': 'req-76760dac-cdce-4691-9f4c-f38c5728b68a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.641654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] The token cache is being invalidate because project 9267a6ccbfa3418a9fd812fc918d8ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:02.642267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.643160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-76760dac-cdce-4691-9f4c-f38c5728b68a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:02.645176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1169/2358] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:02 2022] DELETE /identity/v3/projects/9267a6ccbfa3418a9fd812fc918d8ed8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:02.651409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3618d-8262-4575-8d16-fb0a24e49c2e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:02.651409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3618d-8262-4575-8d16-fb0a24e49c2e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:02.651409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3618d-8262-4575-8d16-fb0a24e49c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:02.672047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-24e3618d-8262-4575-8d16-fb0a24e49c2e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:02.673564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-24e3618d-8262-4575-8d16-fb0a24e49c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:02.743256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1190/2359] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:03:02.753934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede49c3-d3cd-4d4b-b7e8-2ff1e0e2066d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:02.754493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede49c3-d3cd-4d4b-b7e8-2ff1e0e2066d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:02.754854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede49c3-d3cd-4d4b-b7e8-2ff1e0e2066d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:02.778073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-cede49c3-d3cd-4d4b-b7e8-2ff1e0e2066d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:02.779669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-cede49c3-d3cd-4d4b-b7e8-2ff1e0e2066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:02.794896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1170/2360] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:02.799356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3618d-8262-4575-8d16-fb0a24e49c2e None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:02.817372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3618d-8262-4575-8d16-fb0a24e49c2e None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:02.829084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70595c6-5e5d-464c-9a80-9e5e4f7ada6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:02.829084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70595c6-5e5d-464c-9a80-9e5e4f7ada6e admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:02.829084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70595c6-5e5d-464c-9a80-9e5e4f7ada6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:02.829084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70595c6-5e5d-464c-9a80-9e5e4f7ada6e admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:02.830797 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70595c6-5e5d-464c-9a80-9e5e4f7ada6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:03:02.831357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70595c6-5e5d-464c-9a80-9e5e4f7ada6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:02.833429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70595c6-5e5d-464c-9a80-9e5e4f7ada6e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:02.853073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1191/2361] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:03:02.854510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede49c3-d3cd-4d4b-b7e8-2ff1e0e2066d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:02.870042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede49c3-d3cd-4d4b-b7e8-2ff1e0e2066d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:02.894950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c4e9d-2e6e-49e5-929f-f5cf4d58e059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:02.895763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c4e9d-2e6e-49e5-929f-f5cf4d58e059 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:02.901768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c4e9d-2e6e-49e5-929f-f5cf4d58e059 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:02.902231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c4e9d-2e6e-49e5-929f-f5cf4d58e059 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:02.903074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c4e9d-2e6e-49e5-929f-f5cf4d58e059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:02.906765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c4e9d-2e6e-49e5-929f-f5cf4d58e059 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:02.910993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:03.042774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1171/2362] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:03:02 2022] POST /identity/v3/projects => generated 389 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:03:03.046735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70595c6-5e5d-464c-9a80-9e5e4f7ada6e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:03.058569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70595c6-5e5d-464c-9a80-9e5e4f7ada6e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:03.125962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:03.130608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b202c06-ae55-40a6-b90c-5bad20b4519b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:03.130608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b202c06-ae55-40a6-b90c-5bad20b4519b admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:03.130608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b202c06-ae55-40a6-b90c-5bad20b4519b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:03.130608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b202c06-ae55-40a6-b90c-5bad20b4519b admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:03.131041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b202c06-ae55-40a6-b90c-5bad20b4519b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:03.133035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b202c06-ae55-40a6-b90c-5bad20b4519b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:03.238532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1192/2363] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:03:03 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:03:03.244047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c4e9d-2e6e-49e5-929f-f5cf4d58e059 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:03.254556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c4e9d-2e6e-49e5-929f-f5cf4d58e059 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:03.260876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc6dc-6060-43b7-b7fc-ac774bc766fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:03.261491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc6dc-6060-43b7-b7fc-ac774bc766fe admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:03.261744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc6dc-6060-43b7-b7fc-ac774bc766fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:03.262041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc6dc-6060-43b7-b7fc-ac774bc766fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:03.262758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc6dc-6060-43b7-b7fc-ac774bc766fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:03.264934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc6dc-6060-43b7-b7fc-ac774bc766fe admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:03.276287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1172/2364] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:03.280494 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b202c06-ae55-40a6-b90c-5bad20b4519b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:03.291424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b202c06-ae55-40a6-b90c-5bad20b4519b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:03.296100 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fca4d-d859-4e64-8818-99665919df94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:03.296698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fca4d-d859-4e64-8818-99665919df94 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:03.296975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fca4d-d859-4e64-8818-99665919df94 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:03.297247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fca4d-d859-4e64-8818-99665919df94 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:03.297917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fca4d-d859-4e64-8818-99665919df94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:03.299499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fca4d-d859-4e64-8818-99665919df94 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:03.313103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1193/2365] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:03.315909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc6dc-6060-43b7-b7fc-ac774bc766fe admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:03.327006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc6dc-6060-43b7-b7fc-ac774bc766fe admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:03.332208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cfd2c-9a2e-4f17-8631-4126de08352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:03.332894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cfd2c-9a2e-4f17-8631-4126de08352e admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:03.333163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cfd2c-9a2e-4f17-8631-4126de08352e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:03.333432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cfd2c-9a2e-4f17-8631-4126de08352e admin admin] PATH_INFO: `/v3/projects/5854801d4439457294c4a1407d32cae5/users/dc2f8d762f1d47579dc3967e9dc1ced2/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:03.367608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cfd2c-9a2e-4f17-8631-4126de08352e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5854801d4439457294c4a1407d32cae5, user_id=dc2f8d762f1d47579dc3967e9dc1ced2, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:03.370586 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cfd2c-9a2e-4f17-8631-4126de08352e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:03.489453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1173/2366] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:03:03 2022] PUT /identity/v3/projects/5854801d4439457294c4a1407d32cae5/users/dc2f8d762f1d47579dc3967e9dc1ced2/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:03.500190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f018-c677-4903-a04f-e780baedf6b3 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:03.502292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f018-c677-4903-a04f-e780baedf6b3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:03.502640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f018-c677-4903-a04f-e780baedf6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:03.534291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5de4f018-c677-4903-a04f-e780baedf6b3 None None] MFA Rules not processed for user `dc2f8d762f1d47579dc3967e9dc1ced2`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:03.535448 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5de4f018-c677-4903-a04f-e780baedf6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:03.601441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1194/2367] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:03.612311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cfd2c-9a2e-4f17-8631-4126de08352e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:03.614016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-717cfd2c-9a2e-4f17-8631-4126de08352e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:03.671712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cfd2c-9a2e-4f17-8631-4126de08352e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:03.675680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb6fab-9f23-4676-867f-cf2eba2e0b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:03.676277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb6fab-9f23-4676-867f-cf2eba2e0b84 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:03.676554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb6fab-9f23-4676-867f-cf2eba2e0b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:03.676835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb6fab-9f23-4676-867f-cf2eba2e0b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:03.685576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb6fab-9f23-4676-867f-cf2eba2e0b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:03.687045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb6fab-9f23-4676-867f-cf2eba2e0b84 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:03.696532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1174/2368] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:06.045859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4f018-c677-4903-a04f-e780baedf6b3 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:06.047338 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5de4f018-c677-4903-a04f-e780baedf6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:06.117450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4f018-c677-4903-a04f-e780baedf6b3 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:06.122244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:06.124163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.124163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.124163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] PATH_INFO: `/v3/users/6ad81f1d16034a17a655475899bcd74d` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.124589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad81f1d16034a17a655475899bcd74d)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:06.126584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:06.335901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad81f1d16034a17a655475899bcd74d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:06.413916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad81f1d16034a17a655475899bcd74d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:06.422273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad81f1d16034a17a655475899bcd74d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:06.424209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:06.434938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad81f1d16034a17a655475899bcd74d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:06.435678 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad81f1d16034a17a655475899bcd74d'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:06.441884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1195/2369] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:06 2022] DELETE /identity/v3/users/6ad81f1d16034a17a655475899bcd74d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:06.446653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4288050-54c2-4853-b244-f33713dcaefa None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.446653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4288050-54c2-4853-b244-f33713dcaefa None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.449150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4288050-54c2-4853-b244-f33713dcaefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.468783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-e4288050-54c2-4853-b244-f33713dcaefa None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:06.470074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e4288050-54c2-4853-b244-f33713dcaefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:06.544161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1175/2370] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:06.551928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:06.554101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:06.620138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a1ed2-b4b9-4e79-98aa-c85a71746cdd admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:06.623743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5cf76-06cd-40d2-9e92-176227bd7215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:06.624350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5cf76-06cd-40d2-9e92-176227bd7215 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:06.624624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5cf76-06cd-40d2-9e92-176227bd7215 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:06.624892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5cf76-06cd-40d2-9e92-176227bd7215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:06.634179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5cf76-06cd-40d2-9e92-176227bd7215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:06.635597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5cf76-06cd-40d2-9e92-176227bd7215 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:06.645485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1196/2371] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:07.132834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4288050-54c2-4853-b244-f33713dcaefa None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:07.134584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e4288050-54c2-4853-b244-f33713dcaefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:07.147183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4288050-54c2-4853-b244-f33713dcaefa None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:07.151832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:07.152463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:07.152739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:07.153008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] PATH_INFO: `/v3/projects/1b17d36bd8a444248693e01029cd8c9a` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:07.154154 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b17d36bd8a444248693e01029cd8c9a)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:07.157190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:07.298291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:07.299532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b17d36bd8a444248693e01029cd8c9a', 'request_id': 'req-741a4f93-393a-4c8e-91a9-d7cd82157bc1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.306580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b17d36bd8a444248693e01029cd8c9a', 'request_id': 'req-741a4f93-393a-4c8e-91a9-d7cd82157bc1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.373454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b17d36bd8a444248693e01029cd8c9a', 'request_id': 'req-741a4f93-393a-4c8e-91a9-d7cd82157bc1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.385474 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] The token cache is being invalidate because project 1b17d36bd8a444248693e01029cd8c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:07.386216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.386919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:07.388430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1176/2372] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:07 2022] DELETE /identity/v3/projects/1b17d36bd8a444248693e01029cd8c9a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:03:16.640950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=32,keystone:DELETE=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:17.494041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:24.751459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ce927-976f-4a3f-a181-b2c012d11500 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:24.751930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ce927-976f-4a3f-a181-b2c012d11500 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:24.752168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ce927-976f-4a3f-a181-b2c012d11500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:24.775236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4b8ce927-976f-4a3f-a181-b2c012d11500 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:24.777295 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4b8ce927-976f-4a3f-a181-b2c012d11500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:24.841107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1197/2373] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:03:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:03:24.883836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:24.885375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:24.961021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a4f93-393a-4c8e-91a9-d7cd82157bc1 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:24.966178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be53bcf-3060-4877-b206-ada86b0a97df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:24.966798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53bcf-3060-4877-b206-ada86b0a97df service swift] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:24.967118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53bcf-3060-4877-b206-ada86b0a97df service swift] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:24.967390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53bcf-3060-4877-b206-ada86b0a97df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:24.975888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be53bcf-3060-4877-b206-ada86b0a97df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:24.977467 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be53bcf-3060-4877-b206-ada86b0a97df service swift] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:24.987389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1177/2374] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:03:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:03:28.362799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ce927-976f-4a3f-a181-b2c012d11500 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:28.364622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4b8ce927-976f-4a3f-a181-b2c012d11500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:28.463688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ce927-976f-4a3f-a181-b2c012d11500 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:28.469942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:28.470806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:28.471239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:28.471673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] PATH_INFO: `/v3/users/dc2f8d762f1d47579dc3967e9dc1ced2` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:28.475198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2f8d762f1d47579dc3967e9dc1ced2)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:28.479554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:28.701209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2f8d762f1d47579dc3967e9dc1ced2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:28.830237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2f8d762f1d47579dc3967e9dc1ced2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:28.839770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2f8d762f1d47579dc3967e9dc1ced2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:28.855096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2f8d762f1d47579dc3967e9dc1ced2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:28.856181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2f8d762f1d47579dc3967e9dc1ced2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:28.864487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1198/2375] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:28 2022] DELETE /identity/v3/users/dc2f8d762f1d47579dc3967e9dc1ced2 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:28.870383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de1c8f-60dd-4289-9093-1d1d5ea62935 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:28.870383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de1c8f-60dd-4289-9093-1d1d5ea62935 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:28.870998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de1c8f-60dd-4289-9093-1d1d5ea62935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:28.892715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-21de1c8f-60dd-4289-9093-1d1d5ea62935 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:28.894077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-21de1c8f-60dd-4289-9093-1d1d5ea62935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:28.992124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1178/2376] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:29.003161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:29.004672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:29.054625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:29.095454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb424cc1-7219-4456-b444-6d0dfd66a966 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:29.102951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f69737-53b8-479e-8d94-4ee9df5e1fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:29.108285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f69737-53b8-479e-8d94-4ee9df5e1fbe service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:29.108285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f69737-53b8-479e-8d94-4ee9df5e1fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:29.108285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f69737-53b8-479e-8d94-4ee9df5e1fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:29.118310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f69737-53b8-479e-8d94-4ee9df5e1fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:29.121041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f69737-53b8-479e-8d94-4ee9df5e1fbe service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:29.139683 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1199/2377] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:29.908634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de1c8f-60dd-4289-9093-1d1d5ea62935 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:29.910417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-21de1c8f-60dd-4289-9093-1d1d5ea62935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:29.930313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de1c8f-60dd-4289-9093-1d1d5ea62935 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:29.936604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:29.937708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:29.938316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:29.938973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] PATH_INFO: `/v3/projects/5854801d4439457294c4a1407d32cae5` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:29.940332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5854801d4439457294c4a1407d32cae5)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:29.942518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:30.089804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5854801d4439457294c4a1407d32cae5', 'request_id': 'req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:30.097013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5854801d4439457294c4a1407d32cae5', 'request_id': 'req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:30.189063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5854801d4439457294c4a1407d32cae5', 'request_id': 'req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:30.202004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] The token cache is being invalidate because project 5854801d4439457294c4a1407d32cae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:30.202340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:30.202993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:30.204847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1179/2378] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:29 2022] DELETE /identity/v3/projects/5854801d4439457294c4a1407d32cae5 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:39.269336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=15 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:40.198148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:40.293770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb1274-1b1a-4a68-8633-1f1182739acb None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:40.294284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb1274-1b1a-4a68-8633-1f1182739acb None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:40.294405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb1274-1b1a-4a68-8633-1f1182739acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:40.313578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-71bb1274-1b1a-4a68-8633-1f1182739acb None None] MFA Rules not processed for user `d6e9bd6db85b4decbdac1908912d158c`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:40.314830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-71bb1274-1b1a-4a68-8633-1f1182739acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:40.373778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1200/2379] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:41.717052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:41.721087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:41.800066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d9c23-11ea-4bc7-823e-2809c07cc1bd admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:41.804994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb0a8a-0a1b-4a1e-8e91-e46ab38bfc7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:41.805653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb0a8a-0a1b-4a1e-8e91-e46ab38bfc7e service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:41.806077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb0a8a-0a1b-4a1e-8e91-e46ab38bfc7e service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:41.806345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb0a8a-0a1b-4a1e-8e91-e46ab38bfc7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:41.808065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7ffb0a8a-0a1b-4a1e-8e91-e46ab38bfc7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:41.819920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb0a8a-0a1b-4a1e-8e91-e46ab38bfc7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:41.821204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb0a8a-0a1b-4a1e-8e91-e46ab38bfc7e service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:41.831883 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1180/2380] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:44.552460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c76cb-60a3-4991-88fb-d2d5161f71a5 None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:44.552951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c76cb-60a3-4991-88fb-d2d5161f71a5 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:44.553047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c76cb-60a3-4991-88fb-d2d5161f71a5 None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:44.555492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1201/2381] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 17:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:03:44.566367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a08ee-e1e0-4b76-9257-d8f69cfee2e6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:44.566907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a08ee-e1e0-4b76-9257-d8f69cfee2e6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:44.566907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a08ee-e1e0-4b76-9257-d8f69cfee2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:44.601986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-123a08ee-e1e0-4b76-9257-d8f69cfee2e6 None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:44.603796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-123a08ee-e1e0-4b76-9257-d8f69cfee2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:44.699669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1181/2382] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 17:03:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:44.712185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c76cb-60a3-4991-88fb-d2d5161f71a5 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:44.713955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a63c76cb-60a3-4991-88fb-d2d5161f71a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:44.781395 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c76cb-60a3-4991-88fb-d2d5161f71a5 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:44.785426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09e3f8-8e14-4413-82db-62a996be7f5d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73bfe655c6d34918b21acb8748b9156d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:44.786077 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09e3f8-8e14-4413-82db-62a996be7f5d service glance] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:44.786367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09e3f8-8e14-4413-82db-62a996be7f5d service glance] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:44.786612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09e3f8-8e14-4413-82db-62a996be7f5d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:44.799781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09e3f8-8e14-4413-82db-62a996be7f5d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:44.801478 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09e3f8-8e14-4413-82db-62a996be7f5d service glance] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:44.827799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1202/2383] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:45.066396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2973-0c1d-4c10-a7b7-e6f949cf989e None None] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:45.066682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2973-0c1d-4c10-a7b7-e6f949cf989e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:45.066953 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2973-0c1d-4c10-a7b7-e6f949cf989e None None] PATH_INFO: `/` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:45.068658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1182/2384] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 17:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:03:45.073868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86123545-5d19-459b-a7d7-470e323e08fb None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:45.074049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86123545-5d19-459b-a7d7-470e323e08fb None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:45.074467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86123545-5d19-459b-a7d7-470e323e08fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:45.094629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-86123545-5d19-459b-a7d7-470e323e08fb None None] MFA Rules not processed for user `f50ee57e2418417cbe68915ca3f00f76`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:45.095836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-86123545-5d19-459b-a7d7-470e323e08fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:45.154857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1203/2385] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 17:03:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:45.167699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc2973-0c1d-4c10-a7b7-e6f949cf989e None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:45.170048 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-56dc2973-0c1d-4c10-a7b7-e6f949cf989e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:45.184046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc2973-0c1d-4c10-a7b7-e6f949cf989e None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:45.187832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab9106-2ccf-45af-81a8-00fb7f8ac745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:45.188224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab9106-2ccf-45af-81a8-00fb7f8ac745 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:45.188554 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab9106-2ccf-45af-81a8-00fb7f8ac745 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:45.188830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab9106-2ccf-45af-81a8-00fb7f8ac745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:45.212161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab9106-2ccf-45af-81a8-00fb7f8ac745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:45.213852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab9106-2ccf-45af-81a8-00fb7f8ac745 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:45.224125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1183/2386] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:03:45.314413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c58446-b074-479f-a2ce-fc84641acbdf None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:45.318948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c58446-b074-479f-a2ce-fc84641acbdf None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:45.318948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c58446-b074-479f-a2ce-fc84641acbdf None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:45.318948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1204/2387] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 17:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:03:45.322463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45de97-6347-42b6-9178-44356e3d7f8c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:45.322771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45de97-6347-42b6-9178-44356e3d7f8c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:45.323031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45de97-6347-42b6-9178-44356e3d7f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:45.343421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-eb45de97-6347-42b6-9178-44356e3d7f8c None None] MFA Rules not processed for user `54b05cbc41b844d8932ba4c15d960289`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:45.344623 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-eb45de97-6347-42b6-9178-44356e3d7f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:45.354009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1184/2388] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 17:03:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:45.364766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c58446-b074-479f-a2ce-fc84641acbdf None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:45.377186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c58446-b074-479f-a2ce-fc84641acbdf None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:45.381239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2bd82-6485-43f0-980b-04c3d733b329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:45.381968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2bd82-6485-43f0-980b-04c3d733b329 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:45.382890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2bd82-6485-43f0-980b-04c3d733b329 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:45.383215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2bd82-6485-43f0-980b-04c3d733b329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:45.396435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2bd82-6485-43f0-980b-04c3d733b329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:45.398073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2bd82-6485-43f0-980b-04c3d733b329 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:45.411580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1205/2389] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:52.214208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3575-8ef4-4ad5-8b63-bf0b63795069 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:52.215133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3575-8ef4-4ad5-8b63-bf0b63795069 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:52.215405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3575-8ef4-4ad5-8b63-bf0b63795069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:52.235732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-948b3575-8ef4-4ad5-8b63-bf0b63795069 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:52.236968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-948b3575-8ef4-4ad5-8b63-bf0b63795069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:52.303573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1185/2390] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:03:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:52.319265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2bd82-6485-43f0-980b-04c3d733b329 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:52.319265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b1a2bd82-6485-43f0-980b-04c3d733b329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:52.383335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:52.407064 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2bd82-6485-43f0-980b-04c3d733b329 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:52.414744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40e334-6e13-4607-bd1c-a03929f948f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:52.415881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40e334-6e13-4607-bd1c-a03929f948f7 service swift] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:52.416167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40e334-6e13-4607-bd1c-a03929f948f7 service swift] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:52.416557 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40e334-6e13-4607-bd1c-a03929f948f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:52.429887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40e334-6e13-4607-bd1c-a03929f948f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:52.432797 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40e334-6e13-4607-bd1c-a03929f948f7 service swift] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:52.445949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1206/2391] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:03:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:03:53.768299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3575-8ef4-4ad5-8b63-bf0b63795069 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:53.769756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-948b3575-8ef4-4ad5-8b63-bf0b63795069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:53.792352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:53.893415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3575-8ef4-4ad5-8b63-bf0b63795069 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:53.899500 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:53.900147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:53.900421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:53.900691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] PATH_INFO: `/v3/users/5021f06b373543dfa6df4a744e97a33f` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:53.901996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5021f06b373543dfa6df4a744e97a33f)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:53.904052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:54.159670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5021f06b373543dfa6df4a744e97a33f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:54.253340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5021f06b373543dfa6df4a744e97a33f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:54.289624 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5021f06b373543dfa6df4a744e97a33f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:54.302808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5021f06b373543dfa6df4a744e97a33f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:54.310288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5021f06b373543dfa6df4a744e97a33f'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:54.318926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1186/2392] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:03:53 2022] DELETE /identity/v3/users/5021f06b373543dfa6df4a744e97a33f => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:54.323404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be84ac-f352-44e2-9a96-4f26f944159e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:54.323857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be84ac-f352-44e2-9a96-4f26f944159e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:54.324028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be84ac-f352-44e2-9a96-4f26f944159e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:54.345906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-04be84ac-f352-44e2-9a96-4f26f944159e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:54.347395 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-04be84ac-f352-44e2-9a96-4f26f944159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:54.424728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1207/2393] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:03:54.437087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:54.439046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:54.513767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5ad79-cba5-4f20-868b-422697b845d8 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:54.523190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7f1fa-45e9-4e71-9f05-216101ab171b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:54.524393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7f1fa-45e9-4e71-9f05-216101ab171b service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:54.524393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7f1fa-45e9-4e71-9f05-216101ab171b service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:54.524393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7f1fa-45e9-4e71-9f05-216101ab171b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:54.549822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7f1fa-45e9-4e71-9f05-216101ab171b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:54.549822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7f1fa-45e9-4e71-9f05-216101ab171b service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:54.567430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1187/2394] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:03:55.124278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be84ac-f352-44e2-9a96-4f26f944159e None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:55.126360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-04be84ac-f352-44e2-9a96-4f26f944159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:55.161653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be84ac-f352-44e2-9a96-4f26f944159e None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:55.249948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:55.249948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:55.249948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:55.249948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] PATH_INFO: `/v3/projects/f906bc5a327545dcad07f610b6b53c8c` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:55.256748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f906bc5a327545dcad07f610b6b53c8c)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:55.256748 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:55.869553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f906bc5a327545dcad07f610b6b53c8c', 'request_id': 'req-4a5f5c91-7427-4c41-abf2-73bf91d6984a'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:55.876896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f906bc5a327545dcad07f610b6b53c8c', 'request_id': 'req-4a5f5c91-7427-4c41-abf2-73bf91d6984a'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:56.001817 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f906bc5a327545dcad07f610b6b53c8c', 'request_id': 'req-4a5f5c91-7427-4c41-abf2-73bf91d6984a'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:56.011392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] The token cache is being invalidate because project f906bc5a327545dcad07f610b6b53c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:03:56.012595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:56.013739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a5f5c91-7427-4c41-abf2-73bf91d6984a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:03:56.018816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1208/2395] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:03:55 2022] DELETE /identity/v3/projects/f906bc5a327545dcad07f610b6b53c8c => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:03:56.023416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c5e9a-bb0a-4c2e-8003-34e8c95af526 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:56.023985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c5e9a-bb0a-4c2e-8003-34e8c95af526 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:56.023985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c5e9a-bb0a-4c2e-8003-34e8c95af526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:56.051367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dc7c5e9a-bb0a-4c2e-8003-34e8c95af526 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:56.052847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dc7c5e9a-bb0a-4c2e-8003-34e8c95af526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:56.078001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:56.120643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1188/2396] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:56.128844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ff2ae-687e-4ef6-a32b-292e9eca1648 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:56.129242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ff2ae-687e-4ef6-a32b-292e9eca1648 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:56.129585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ff2ae-687e-4ef6-a32b-292e9eca1648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:56.155215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-379ff2ae-687e-4ef6-a32b-292e9eca1648 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:56.156744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-379ff2ae-687e-4ef6-a32b-292e9eca1648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:56.165573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1209/2397] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:56.170577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c5e9a-bb0a-4c2e-8003-34e8c95af526 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:56.184072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c5e9a-bb0a-4c2e-8003-34e8c95af526 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:56.185738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7b4fd-f635-4529-8a8f-83fe7e8967b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:56.186498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7b4fd-f635-4529-8a8f-83fe7e8967b5 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:56.186799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7b4fd-f635-4529-8a8f-83fe7e8967b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:56.187061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7b4fd-f635-4529-8a8f-83fe7e8967b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:56.187702 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7b4fd-f635-4529-8a8f-83fe7e8967b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:03:56.187995 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7b4fd-f635-4529-8a8f-83fe7e8967b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:56.189603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7b4fd-f635-4529-8a8f-83fe7e8967b5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:56.202209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1189/2398] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:03:56.206895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ff2ae-687e-4ef6-a32b-292e9eca1648 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:56.217701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ff2ae-687e-4ef6-a32b-292e9eca1648 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:56.222608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d302f4-2463-499e-a221-d3bb0f79bbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:56.223250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d302f4-2463-499e-a221-d3bb0f79bbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:56.223527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d302f4-2463-499e-a221-d3bb0f79bbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:56.223839 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d302f4-2463-499e-a221-d3bb0f79bbb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:56.224455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d302f4-2463-499e-a221-d3bb0f79bbb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:56.226459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d302f4-2463-499e-a221-d3bb0f79bbb4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:56.359864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1210/2399] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:03:56 2022] POST /identity/v3/projects => generated 399 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:56.364275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7b4fd-f635-4529-8a8f-83fe7e8967b5 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:56.391657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7b4fd-f635-4529-8a8f-83fe7e8967b5 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:56.469092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038dcd0-8ed2-4d76-be48-fc4855a6e5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:56.469807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038dcd0-8ed2-4d76-be48-fc4855a6e5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:56.474169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038dcd0-8ed2-4d76-be48-fc4855a6e5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:56.474491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038dcd0-8ed2-4d76-be48-fc4855a6e5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:56.475351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038dcd0-8ed2-4d76-be48-fc4855a6e5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:56.477424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038dcd0-8ed2-4d76-be48-fc4855a6e5a7 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:56.601020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1190/2400] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:03:56 2022] POST /identity/v3/users => generated 300 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:03:56.605106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d302f4-2463-499e-a221-d3bb0f79bbb4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:56.615784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d302f4-2463-499e-a221-d3bb0f79bbb4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:56.620415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb716-772d-465b-9138-10ee08381d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:56.621011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb716-772d-465b-9138-10ee08381d85 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:56.621289 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb716-772d-465b-9138-10ee08381d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:56.621582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb716-772d-465b-9138-10ee08381d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:56.622281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffb716-772d-465b-9138-10ee08381d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:56.623952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffb716-772d-465b-9138-10ee08381d85 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:56.634204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1211/2401] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:56.640286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038dcd0-8ed2-4d76-be48-fc4855a6e5a7 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:56.650514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038dcd0-8ed2-4d76-be48-fc4855a6e5a7 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:56.655688 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f7b3-87ad-4a0e-acb2-866e9ce8957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:56.656283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f7b3-87ad-4a0e-acb2-866e9ce8957c admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:56.656564 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f7b3-87ad-4a0e-acb2-866e9ce8957c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:56.656836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f7b3-87ad-4a0e-acb2-866e9ce8957c admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:56.657475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77f7b3-87ad-4a0e-acb2-866e9ce8957c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:56.659276 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77f7b3-87ad-4a0e-acb2-866e9ce8957c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:56.671581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1191/2402] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:03:56.675855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb716-772d-465b-9138-10ee08381d85 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:56.687196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb716-772d-465b-9138-10ee08381d85 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:56.692217 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c570-f854-408a-a202-8dc060ef8394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:56.696871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c570-f854-408a-a202-8dc060ef8394 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:56.696871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c570-f854-408a-a202-8dc060ef8394 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:56.696871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c570-f854-408a-a202-8dc060ef8394 admin admin] PATH_INFO: `/v3/projects/b24d1f6e7ee84adc8792d9cf39127f2a/users/36c141b2b78a4882ba115b8c5287b047/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:56.710982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13c570-f854-408a-a202-8dc060ef8394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b24d1f6e7ee84adc8792d9cf39127f2a, user_id=36c141b2b78a4882ba115b8c5287b047, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:56.713649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13c570-f854-408a-a202-8dc060ef8394 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:56.716222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:03:56.819669 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1212/2403] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:03:56 2022] PUT /identity/v3/projects/b24d1f6e7ee84adc8792d9cf39127f2a/users/36c141b2b78a4882ba115b8c5287b047/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:03:56.895330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0496c0-004b-4134-b16c-0cb3cd8d7a77 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:56.895638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0496c0-004b-4134-b16c-0cb3cd8d7a77 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:56.895884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0496c0-004b-4134-b16c-0cb3cd8d7a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:56.928428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9a0496c0-004b-4134-b16c-0cb3cd8d7a77 None None] MFA Rules not processed for user `36c141b2b78a4882ba115b8c5287b047`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:03:56.929690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9a0496c0-004b-4134-b16c-0cb3cd8d7a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:57.002539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1192/2404] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:03:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:03:57.015149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c570-f854-408a-a202-8dc060ef8394 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:03:57.017384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ba13c570-f854-408a-a202-8dc060ef8394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:03:57.125330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c570-f854-408a-a202-8dc060ef8394 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:03:57.130469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93078c30-d92b-4983-879b-209dcbc35558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:03:57.131502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93078c30-d92b-4983-879b-209dcbc35558 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:03:57.131812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93078c30-d92b-4983-879b-209dcbc35558 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:03:57.132132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93078c30-d92b-4983-879b-209dcbc35558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:03:57.143118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93078c30-d92b-4983-879b-209dcbc35558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:03:57.145463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93078c30-d92b-4983-879b-209dcbc35558 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:03:57.157457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1213/2405] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:03:57 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:04:07.002782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:07.151732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:07.346462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0496c0-004b-4134-b16c-0cb3cd8d7a77 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:04:07.409606 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0496c0-004b-4134-b16c-0cb3cd8d7a77 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:04:07.413984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:04:07.416102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:07.416102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:07.416102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] PATH_INFO: `/v3/users/36c141b2b78a4882ba115b8c5287b047` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:07.416507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36c141b2b78a4882ba115b8c5287b047)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:04:07.418447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:04:07.659571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36c141b2b78a4882ba115b8c5287b047'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:07.751097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36c141b2b78a4882ba115b8c5287b047'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:07.765255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36c141b2b78a4882ba115b8c5287b047'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:07.773077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36c141b2b78a4882ba115b8c5287b047'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:07.773985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36c141b2b78a4882ba115b8c5287b047'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:07.780578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1193/2406] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:04:07 2022] DELETE /identity/v3/users/36c141b2b78a4882ba115b8c5287b047 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:04:07.792934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c4239-c5e2-49aa-a52e-4f35bba1f004 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:07.793331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c4239-c5e2-49aa-a52e-4f35bba1f004 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:07.793614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c4239-c5e2-49aa-a52e-4f35bba1f004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:07.815326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-295c4239-c5e2-49aa-a52e-4f35bba1f004 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:04:07.817617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-295c4239-c5e2-49aa-a52e-4f35bba1f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:07.880747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1214/2407] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:04:07.890765 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:04:07.892279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:07.951257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1eeb3-23ef-4dcb-b3b2-959b08da5667 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:04:07.954977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b3727-ea18-4d05-a763-d21f22a8c5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:04:07.955582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b3727-ea18-4d05-a763-d21f22a8c5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:07.955858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b3727-ea18-4d05-a763-d21f22a8c5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:07.956126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b3727-ea18-4d05-a763-d21f22a8c5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:07.965260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b3727-ea18-4d05-a763-d21f22a8c5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:04:07.969495 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b3727-ea18-4d05-a763-d21f22a8c5a5 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:04:07.978712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1194/2408] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:04:08.493619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c4239-c5e2-49aa-a52e-4f35bba1f004 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:04:08.495200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-295c4239-c5e2-49aa-a52e-4f35bba1f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:08.511417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c4239-c5e2-49aa-a52e-4f35bba1f004 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:04:08.516739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:04:08.517367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:08.517639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:08.517960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] PATH_INFO: `/v3/projects/b24d1f6e7ee84adc8792d9cf39127f2a` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:08.523459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b24d1f6e7ee84adc8792d9cf39127f2a)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:04:08.525299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:04:08.632215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b24d1f6e7ee84adc8792d9cf39127f2a', 'request_id': 'req-ac1f7f8a-4869-4389-9a13-b1903915f684'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:08.639221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b24d1f6e7ee84adc8792d9cf39127f2a', 'request_id': 'req-ac1f7f8a-4869-4389-9a13-b1903915f684'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:08.720099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b24d1f6e7ee84adc8792d9cf39127f2a', 'request_id': 'req-ac1f7f8a-4869-4389-9a13-b1903915f684'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:08.731836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] The token cache is being invalidate because project b24d1f6e7ee84adc8792d9cf39127f2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:04:08.732949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:08.732949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:04:08.734910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1215/2409] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:04:08 2022] DELETE /identity/v3/projects/b24d1f6e7ee84adc8792d9cf39127f2a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:04:17.974387 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:18.728620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:29.185352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41474d3a-cf51-498c-8545-a4a505697255 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:29.185851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41474d3a-cf51-498c-8545-a4a505697255 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:29.186124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41474d3a-cf51-498c-8545-a4a505697255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:29.205570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-41474d3a-cf51-498c-8545-a4a505697255 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:04:29.208377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-41474d3a-cf51-498c-8545-a4a505697255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:29.270421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1195/2410] 10.222.0.57 () {58 vars in 967 bytes} [Mon Aug 8 17:04:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:04:29.293508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:04:29.294993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:29.374431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f7f8a-4869-4389-9a13-b1903915f684 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:04:29.379553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539656-d116-4355-be29-488770cf9266 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:04:29.380194 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539656-d116-4355-be29-488770cf9266 service swift] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:29.380472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539656-d116-4355-be29-488770cf9266 service swift] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:29.380746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539656-d116-4355-be29-488770cf9266 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:29.396154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539656-d116-4355-be29-488770cf9266 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:04:29.397775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539656-d116-4355-be29-488770cf9266 service swift] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:04:29.407157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1216/2411] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:04:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:04:39.266938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=25 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:39.403233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=32 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:41.746631 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba1dcd-f070-42eb-8b6a-f5cbf14f6d11 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:41.747163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba1dcd-f070-42eb-8b6a-f5cbf14f6d11 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:41.747276 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba1dcd-f070-42eb-8b6a-f5cbf14f6d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:41.766058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ceba1dcd-f070-42eb-8b6a-f5cbf14f6d11 None None] MFA Rules not processed for user `5a7ed5f2559a449fbe0b30bfeabd90df`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:04:41.767248 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ceba1dcd-f070-42eb-8b6a-f5cbf14f6d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:41.828786 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1196/2412] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:04:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:04:46.195640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539656-d116-4355-be29-488770cf9266 service swift] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:04:46.197170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2e539656-d116-4355-be29-488770cf9266 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:46.255883 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539656-d116-4355-be29-488770cf9266 service swift] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:04:46.259459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede0a91-9078-439e-bc73-92aeedf682d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:04:46.260060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede0a91-9078-439e-bc73-92aeedf682d9 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:04:46.260331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede0a91-9078-439e-bc73-92aeedf682d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:04:46.260595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede0a91-9078-439e-bc73-92aeedf682d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:04:46.262350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-eede0a91-9078-439e-bc73-92aeedf682d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:04:46.328890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede0a91-9078-439e-bc73-92aeedf682d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:04:46.330378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede0a91-9078-439e-bc73-92aeedf682d9 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:04:46.340291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1217/2413] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:04:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:04:51.820424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=25 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:04:56.335799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=50 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:05:00.044840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6903f-f4cd-4aa1-98a0-023a0c842912 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:05:00.045473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6903f-f4cd-4aa1-98a0-023a0c842912 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:05:00.045473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6903f-f4cd-4aa1-98a0-023a0c842912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:05:00.066622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-eab6903f-f4cd-4aa1-98a0-023a0c842912 None None] MFA Rules not processed for user `e941c29e3df944a08f18f5e029a28977`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:05:00.067860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-eab6903f-f4cd-4aa1-98a0-023a0c842912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:05:00.137853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1197/2414] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:05:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:05:06.409189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede0a91-9078-439e-bc73-92aeedf682d9 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:05:06.420619 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede0a91-9078-439e-bc73-92aeedf682d9 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:05:06.425391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:05:06.426049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:05:06.426328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:05:06.426596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:05:06.428770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:05:06.511258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:05:06.512977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:05:06.525596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1218/2415] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:05:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:05:07.664371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6903f-f4cd-4aa1-98a0-023a0c842912 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:05:07.666304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-eab6903f-f4cd-4aa1-98a0-023a0c842912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:05:07.678881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6903f-f4cd-4aa1-98a0-023a0c842912 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:05:07.684374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:05:07.684990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:05:07.685270 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:05:07.685534 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:05:07.687861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:05:07.702484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:05:07.703925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:05:07.715300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1198/2416] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:05:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:05:16.519957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=34 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:05:17.710280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=31 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:05:22.786157 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:05:22.787731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:05:22.802498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c47ff-936f-4b47-8a8b-9ee2c1967ecf service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:05:22.808181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd453e29-9534-4320-92c9-f3ef341c3034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:05:22.808777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd453e29-9534-4320-92c9-f3ef341c3034 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:05:22.809053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd453e29-9534-4320-92c9-f3ef341c3034 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:05:22.809321 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd453e29-9534-4320-92c9-f3ef341c3034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:05:22.812010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fd453e29-9534-4320-92c9-f3ef341c3034 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:05:22.869641 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd453e29-9534-4320-92c9-f3ef341c3034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:05:22.871185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd453e29-9534-4320-92c9-f3ef341c3034 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:05:22.881053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1219/2417] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:05:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:05:32.876007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=28 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:05:34.014248 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:05:34.015658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:05:34.029736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa52371-8e69-478c-8c43-1e5893198bd0 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:05:34.036042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c4bc9-762f-4207-9e24-b07668ae55c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:05:34.036730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c4bc9-762f-4207-9e24-b07668ae55c5 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:05:34.037227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c4bc9-762f-4207-9e24-b07668ae55c5 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:05:34.037730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c4bc9-762f-4207-9e24-b07668ae55c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:05:34.041028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dc2c4bc9-762f-4207-9e24-b07668ae55c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:05:34.105315 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c4bc9-762f-4207-9e24-b07668ae55c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:05:34.107051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c4bc9-762f-4207-9e24-b07668ae55c5 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:05:34.118811 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1199/2418] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:05:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:05:44.112322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=28 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:06:29.896542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd453e29-9534-4320-92c9-f3ef341c3034 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:06:29.906773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd453e29-9534-4320-92c9-f3ef341c3034 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:06:29.910796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e975b9-18fb-486c-a2ac-f3d513f28018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:06:29.911425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e975b9-18fb-486c-a2ac-f3d513f28018 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:06:29.911700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e975b9-18fb-486c-a2ac-f3d513f28018 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:06:29.911974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e975b9-18fb-486c-a2ac-f3d513f28018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:06:29.914062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-32e975b9-18fb-486c-a2ac-f3d513f28018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:06:29.924283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e975b9-18fb-486c-a2ac-f3d513f28018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:06:29.925701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e975b9-18fb-486c-a2ac-f3d513f28018 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:06:29.933750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1220/2419] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:06:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:06:32.204177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c4bc9-762f-4207-9e24-b07668ae55c5 service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:06:32.217558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c4bc9-762f-4207-9e24-b07668ae55c5 service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:06:32.224654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a49419-9317-48db-8d68-247469b96a7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:06:32.229736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a49419-9317-48db-8d68-247469b96a7c service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:06:32.229736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a49419-9317-48db-8d68-247469b96a7c service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:06:32.229736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a49419-9317-48db-8d68-247469b96a7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:06:32.229736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-19a49419-9317-48db-8d68-247469b96a7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:06:32.260717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a49419-9317-48db-8d68-247469b96a7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:06:32.262752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a49419-9317-48db-8d68-247469b96a7c service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:06:32.277208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1200/2420] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:06:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:06:39.930266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=6 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:06:42.271903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=6 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:06:54.651623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a632f-f11f-4982-ba25-cd587c5f03e6 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:06:54.652216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a632f-f11f-4982-ba25-cd587c5f03e6 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:06:54.652216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a632f-f11f-4982-ba25-cd587c5f03e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:06:54.671296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8a5a632f-f11f-4982-ba25-cd587c5f03e6 None None] MFA Rules not processed for user `d6e9bd6db85b4decbdac1908912d158c`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:06:54.672432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8a5a632f-f11f-4982-ba25-cd587c5f03e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:06:54.679268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1221/2421] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:06:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:07:00.278734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a49419-9317-48db-8d68-247469b96a7c service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:07:00.291092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a49419-9317-48db-8d68-247469b96a7c service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:07:00.302129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:07:00.302742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:07:00.303016 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:07:00.303278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:07:00.305530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:07:00.316511 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:07:00.318124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:07:00.327148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1201/2422] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:07:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:07:01.141388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a632f-f11f-4982-ba25-cd587c5f03e6 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:07:01.154801 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a632f-f11f-4982-ba25-cd587c5f03e6 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:07:01.160860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dbabf-e27e-4d2a-b875-4d9f0e8722b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:07:01.161477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dbabf-e27e-4d2a-b875-4d9f0e8722b5 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:07:01.161780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dbabf-e27e-4d2a-b875-4d9f0e8722b5 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:07:01.162192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822dbabf-e27e-4d2a-b875-4d9f0e8722b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:07:01.164223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-822dbabf-e27e-4d2a-b875-4d9f0e8722b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:07:01.177463 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822dbabf-e27e-4d2a-b875-4d9f0e8722b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:07:01.179167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822dbabf-e27e-4d2a-b875-4d9f0e8722b5 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:07:01.189726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1222/2423] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:07:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:07:10.322958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=6 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:07:11.185962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=9 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:07:14.791569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:07:14.792955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:07:14.848290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92f091-f34b-4ea9-a045-f65e5de9e0cf service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:07:14.851774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11f55-0294-47f7-a170-ab7a52114484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:07:14.852389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11f55-0294-47f7-a170-ab7a52114484 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:07:14.852666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11f55-0294-47f7-a170-ab7a52114484 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:07:14.852935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11f55-0294-47f7-a170-ab7a52114484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:07:14.854831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9dc11f55-0294-47f7-a170-ab7a52114484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:07:14.866986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc11f55-0294-47f7-a170-ab7a52114484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:07:14.868407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc11f55-0294-47f7-a170-ab7a52114484 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:07:14.877258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1202/2424] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:07:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:07:24.873235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=26 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:07:39.959069 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dbabf-e27e-4d2a-b875-4d9f0e8722b5 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:07:39.971287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822dbabf-e27e-4d2a-b875-4d9f0e8722b5 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:07:39.977730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:07:39.978376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:07:39.978657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:07:39.987031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:07:39.987031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:07:39.995492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:07:39.997150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:07:40.008603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1223/2425] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:07:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:07:50.003660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=6 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:07:50.406330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11f55-0294-47f7-a170-ab7a52114484 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:07:50.419420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11f55-0294-47f7-a170-ab7a52114484 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:07:50.424766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:07:50.425354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:07:50.425633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:07:50.427239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:07:50.427768 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:07:50.485987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:07:50.487569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:07:50.496930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1203/2426] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:07:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:08:00.492351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=28 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:09:08.058497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:09:08.058497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:09:08.221128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e0838-3b80-47a9-aeaa-ba493516035c service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:09:08.226327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb39ff3-3dfe-46ab-8af1-b9e73fcb207f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:09:08.226930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb39ff3-3dfe-46ab-8af1-b9e73fcb207f service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:09:08.227205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb39ff3-3dfe-46ab-8af1-b9e73fcb207f service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:09:08.227470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb39ff3-3dfe-46ab-8af1-b9e73fcb207f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:09:08.229310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cdb39ff3-3dfe-46ab-8af1-b9e73fcb207f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:09:08.241854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb39ff3-3dfe-46ab-8af1-b9e73fcb207f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:09:08.243363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb39ff3-3dfe-46ab-8af1-b9e73fcb207f service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:09:08.254327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1224/2427] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:09:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:09:18.249128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=63 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:10:32.519134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:32.520541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:10:32.574732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe359be-eecb-4e15-b1f3-85c592a61039 service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:32.578338 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a815187-83b6-42db-ab56-e7a4f0358f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:32.578935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a815187-83b6-42db-ab56-e7a4f0358f74 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:32.579210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a815187-83b6-42db-ab56-e7a4f0358f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:32.579481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a815187-83b6-42db-ab56-e7a4f0358f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:32.597250 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a815187-83b6-42db-ab56-e7a4f0358f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:32.598846 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a815187-83b6-42db-ab56-e7a4f0358f74 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:32.620630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1204/2428] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:10:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:10:32.627744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb39ff3-3dfe-46ab-8af1-b9e73fcb207f service placement] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:32.638055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb39ff3-3dfe-46ab-8af1-b9e73fcb207f service placement] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:32.641682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83b6a0-fff4-416b-9b08-6ca281aaba25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:32.642769 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83b6a0-fff4-416b-9b08-6ca281aaba25 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:32.643040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83b6a0-fff4-416b-9b08-6ca281aaba25 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:32.643306 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83b6a0-fff4-416b-9b08-6ca281aaba25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:32.651903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83b6a0-fff4-416b-9b08-6ca281aaba25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:32.653308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83b6a0-fff4-416b-9b08-6ca281aaba25 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:32.683972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1225/2429] 10.222.0.57 () {60 vars in 1396 bytes} [Mon Aug 8 17:10:32 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:10:32.820051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a815187-83b6-42db-ab56-e7a4f0358f74 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:32.828562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a815187-83b6-42db-ab56-e7a4f0358f74 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:32.832155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8a887-f6a0-4b9e-9ba5-9cc1c0a499d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:32.832738 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8a887-f6a0-4b9e-9ba5-9cc1c0a499d8 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:32.833012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8a887-f6a0-4b9e-9ba5-9cc1c0a499d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:32.833276 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8a887-f6a0-4b9e-9ba5-9cc1c0a499d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:32.843382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8a887-f6a0-4b9e-9ba5-9cc1c0a499d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:32.844795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8a887-f6a0-4b9e-9ba5-9cc1c0a499d8 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:32.856114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1205/2430] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:10:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:10:34.080596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83b6a0-fff4-416b-9b08-6ca281aaba25 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:10:34.093151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83b6a0-fff4-416b-9b08-6ca281aaba25 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:10:34.099209 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165343d0-28e7-49cd-b432-d9d343a63f0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:10:34.099806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165343d0-28e7-49cd-b432-d9d343a63f0c service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:10:34.100080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165343d0-28e7-49cd-b432-d9d343a63f0c service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:10:34.100345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165343d0-28e7-49cd-b432-d9d343a63f0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:10:34.108506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165343d0-28e7-49cd-b432-d9d343a63f0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:10:34.110151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165343d0-28e7-49cd-b432-d9d343a63f0c service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:10:34.138325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1226/2431] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:10:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:10:42.851450 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=37 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:10:44.135763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=26 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:12:13.637848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8a887-f6a0-4b9e-9ba5-9cc1c0a499d8 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:12:13.656594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8a887-f6a0-4b9e-9ba5-9cc1c0a499d8 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:12:13.660633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41be10-8d03-4272-8860-452d75f28958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:12:13.661222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41be10-8d03-4272-8860-452d75f28958 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:12:13.661494 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41be10-8d03-4272-8860-452d75f28958 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:12:13.661759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41be10-8d03-4272-8860-452d75f28958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:12:13.668981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41be10-8d03-4272-8860-452d75f28958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:12:13.670436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41be10-8d03-4272-8860-452d75f28958 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:12:13.678937 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1206/2432] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:12:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:12:23.674773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=10 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:12:33.123052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165343d0-28e7-49cd-b432-d9d343a63f0c service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:12:33.134462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165343d0-28e7-49cd-b432-d9d343a63f0c service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:12:33.140629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a864c-2c5d-4b17-90f8-9ebf8ee9060f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:12:33.141363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a864c-2c5d-4b17-90f8-9ebf8ee9060f service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:12:33.141645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a864c-2c5d-4b17-90f8-9ebf8ee9060f service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:12:33.142004 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a864c-2c5d-4b17-90f8-9ebf8ee9060f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:12:33.151006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a864c-2c5d-4b17-90f8-9ebf8ee9060f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:12:33.152577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a864c-2c5d-4b17-90f8-9ebf8ee9060f service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:12:33.165197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41be10-8d03-4272-8860-452d75f28958 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:12:33.180130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41be10-8d03-4272-8860-452d75f28958 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:12:33.183330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1227/2433] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:12:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:12:33.189140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a15067-bc26-440e-8b70-14cdc9700632 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:12:33.189839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a15067-bc26-440e-8b70-14cdc9700632 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:12:33.190183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a15067-bc26-440e-8b70-14cdc9700632 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:12:33.190514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a15067-bc26-440e-8b70-14cdc9700632 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:12:33.199326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a15067-bc26-440e-8b70-14cdc9700632 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:12:33.200802 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a15067-bc26-440e-8b70-14cdc9700632 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:12:33.230285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1207/2434] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:12:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:12:43.180595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=13 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:12:43.227694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=13 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:13:37.460812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a864c-2c5d-4b17-90f8-9ebf8ee9060f service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:13:37.470599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a864c-2c5d-4b17-90f8-9ebf8ee9060f service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:13:37.474190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2556-4b4d-4ac3-86a7-f633dd97fff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:13:37.474803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2556-4b4d-4ac3-86a7-f633dd97fff9 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:13:37.475075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2556-4b4d-4ac3-86a7-f633dd97fff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:13:37.475347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2556-4b4d-4ac3-86a7-f633dd97fff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:13:37.483899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f2556-4b4d-4ac3-86a7-f633dd97fff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:13:37.485309 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f2556-4b4d-4ac3-86a7-f633dd97fff9 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:13:37.512512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1228/2435] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:13:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:13:47.509907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=13 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:14:15.030235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a15067-bc26-440e-8b70-14cdc9700632 service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:14:15.031614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a4a15067-bc26-440e-8b70-14cdc9700632 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:14:15.046687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a15067-bc26-440e-8b70-14cdc9700632 service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:14:15.051469 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1f016-3f82-4cba-83f5-935ba45baa8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:14:15.052090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1f016-3f82-4cba-83f5-935ba45baa8a service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:14:15.052361 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1f016-3f82-4cba-83f5-935ba45baa8a service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:14:15.052628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1f016-3f82-4cba-83f5-935ba45baa8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:14:15.061259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1f016-3f82-4cba-83f5-935ba45baa8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:14:15.062852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1f016-3f82-4cba-83f5-935ba45baa8a service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:14:15.072572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1208/2436] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:14:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:14:25.068404 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=6 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:15:37.548976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2556-4b4d-4ac3-86a7-f633dd97fff9 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:15:37.558047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2556-4b4d-4ac3-86a7-f633dd97fff9 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:15:37.561612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a26f6-8098-49be-bf45-6d5c64ad4dce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:15:37.562241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a26f6-8098-49be-bf45-6d5c64ad4dce service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:15:37.562518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a26f6-8098-49be-bf45-6d5c64ad4dce service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:15:37.562785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a26f6-8098-49be-bf45-6d5c64ad4dce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:15:37.564599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cd9a26f6-8098-49be-bf45-6d5c64ad4dce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:15:37.649707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a26f6-8098-49be-bf45-6d5c64ad4dce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:15:37.651148 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a26f6-8098-49be-bf45-6d5c64ad4dce service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:15:37.661906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1229/2437] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:15:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:15:37.716096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1f016-3f82-4cba-83f5-935ba45baa8a service placement] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:15:37.725318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1f016-3f82-4cba-83f5-935ba45baa8a service placement] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:15:37.729649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a025ff-ece8-41a4-a470-eeac65b575fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:15:37.730332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a025ff-ece8-41a4-a470-eeac65b575fe service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:15:37.730670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a025ff-ece8-41a4-a470-eeac65b575fe service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:15:37.731020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a025ff-ece8-41a4-a470-eeac65b575fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:15:37.733159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a9a025ff-ece8-41a4-a470-eeac65b575fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:15:37.746622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a025ff-ece8-41a4-a470-eeac65b575fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:15:37.748084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a025ff-ece8-41a4-a470-eeac65b575fe service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:15:37.758778 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1209/2438] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:15:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:15:37.765697 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a26f6-8098-49be-bf45-6d5c64ad4dce service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:15:37.775255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a26f6-8098-49be-bf45-6d5c64ad4dce service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:15:37.778906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:15:37.779497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:15:37.779770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:15:37.780036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:15:37.781872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:15:37.852956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:15:37.854408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:15:37.863202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1230/2439] 10.222.0.57 () {60 vars in 1396 bytes} [Mon Aug 8 17:15:37 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:15:37.910807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a025ff-ece8-41a4-a470-eeac65b575fe service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:15:37.919378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a025ff-ece8-41a4-a470-eeac65b575fe service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:15:37.923062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:15:37.923595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:15:37.923870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:15:37.924138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:15:37.925977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:15:38.001108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:15:38.002560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:15:38.012058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1210/2440] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:15:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:15:47.859034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=72 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:15:48.008000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=39 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:16:36.517196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:16:36.518901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:16:36.536347 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134fae8-119d-43c6-9895-feef7e34ddfc service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:16:36.541880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3b2e2-f5c7-404d-8732-533a47d62593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:16:36.542524 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3b2e2-f5c7-404d-8732-533a47d62593 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:16:36.542838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3b2e2-f5c7-404d-8732-533a47d62593 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:16:36.543311 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3b2e2-f5c7-404d-8732-533a47d62593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:16:36.552323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3b2e2-f5c7-404d-8732-533a47d62593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:16:36.553919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3b2e2-f5c7-404d-8732-533a47d62593 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:16:36.564727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1231/2441] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:16:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:16:46.560464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=6 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:17:19.615453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:17:19.617033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:17:19.685535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03ee90-ed38-40bb-b876-646d6579679c service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:17:19.689087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafb7cc-dd09-4e6c-8f40-1049fbda1e4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:17:19.689670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafb7cc-dd09-4e6c-8f40-1049fbda1e4d service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:17:19.689974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafb7cc-dd09-4e6c-8f40-1049fbda1e4d service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:17:19.690241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafb7cc-dd09-4e6c-8f40-1049fbda1e4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:17:19.692031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8fafb7cc-dd09-4e6c-8f40-1049fbda1e4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:17:19.700963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafb7cc-dd09-4e6c-8f40-1049fbda1e4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:17:19.702925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafb7cc-dd09-4e6c-8f40-1049fbda1e4d service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:17:19.710510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1211/2442] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:17:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:17:29.707143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=31 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:17:38.224758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3b2e2-f5c7-404d-8732-533a47d62593 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:17:38.224758 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-02a3b2e2-f5c7-404d-8732-533a47d62593 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:17:38.264859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3b2e2-f5c7-404d-8732-533a47d62593 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:17:38.268506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939efd0-e6d4-4888-8389-8e14dc240177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:17:38.269108 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939efd0-e6d4-4888-8389-8e14dc240177 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:17:38.269388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939efd0-e6d4-4888-8389-8e14dc240177 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:17:38.269936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939efd0-e6d4-4888-8389-8e14dc240177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:17:38.272068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b939efd0-e6d4-4888-8389-8e14dc240177 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:17:38.346739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939efd0-e6d4-4888-8389-8e14dc240177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:17:38.348117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939efd0-e6d4-4888-8389-8e14dc240177 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:17:38.356990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1232/2443] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:17:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:17:38.405685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafb7cc-dd09-4e6c-8f40-1049fbda1e4d service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:17:38.414803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafb7cc-dd09-4e6c-8f40-1049fbda1e4d service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:17:38.418277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:17:38.423923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:17:38.423923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:17:38.423923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:17:38.423923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:17:38.491640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:17:38.493341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:17:38.504056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1212/2444] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:17:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:17:39.155914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939efd0-e6d4-4888-8389-8e14dc240177 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:17:39.171655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939efd0-e6d4-4888-8389-8e14dc240177 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:17:39.178278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:17:39.178898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:17:39.179178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:17:39.179447 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:17:39.181401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:17:39.195105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:17:39.197722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:17:39.208007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1233/2445] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:17:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:17:48.498463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=33 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:17:49.203773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=59 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:18:37.645208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:18:37.646978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:18:37.662757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02a1ff-316f-4957-9d46-ad007a8e0ef8 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:18:37.668212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:18:37.668799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:18:37.669074 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:18:37.669340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:18:37.671047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:18:37.738274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:18:37.739847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:18:37.749654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1213/2446] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:18:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:18:47.744653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=33 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:19:18.980876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:19:18.983252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:19:19.104676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccd091-fdb9-4662-912e-ddb96cc57c95 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:19:19.109723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f878c5-a019-4775-a381-bf56a67fa1a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:19:19.110351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f878c5-a019-4775-a381-bf56a67fa1a9 service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:19:19.110631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f878c5-a019-4775-a381-bf56a67fa1a9 service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:19:19.110903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f878c5-a019-4775-a381-bf56a67fa1a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:19:19.112798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b1f878c5-a019-4775-a381-bf56a67fa1a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:19:19.123926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f878c5-a019-4775-a381-bf56a67fa1a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:19:19.125496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f878c5-a019-4775-a381-bf56a67fa1a9 service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:19:19.135818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1234/2447] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:19:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:19:29.131546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=54 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:20:36.621704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:20:36.623352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:20:36.653941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e12c0-3781-47e3-a7ca-5f95fab19c02 service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:20:36.657484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c74c18-1add-47a7-8c46-9aedda15d906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:20:36.658097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c74c18-1add-47a7-8c46-9aedda15d906 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:20:36.658373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c74c18-1add-47a7-8c46-9aedda15d906 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:20:36.658636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c74c18-1add-47a7-8c46-9aedda15d906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:20:36.675742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c74c18-1add-47a7-8c46-9aedda15d906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:20:36.677182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c74c18-1add-47a7-8c46-9aedda15d906 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:20:36.697845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1214/2448] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:20:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:20:36.737422 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f878c5-a019-4775-a381-bf56a67fa1a9 service placement] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:20:36.761747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f878c5-a019-4775-a381-bf56a67fa1a9 service placement] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:20:36.767353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da75d9c-d772-4e4d-a7ce-d1c4cfd61614 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:20:36.767948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da75d9c-d772-4e4d-a7ce-d1c4cfd61614 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:20:36.768221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da75d9c-d772-4e4d-a7ce-d1c4cfd61614 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:20:36.768486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da75d9c-d772-4e4d-a7ce-d1c4cfd61614 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:20:36.776421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da75d9c-d772-4e4d-a7ce-d1c4cfd61614 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:20:36.778016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da75d9c-d772-4e4d-a7ce-d1c4cfd61614 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:20:36.805827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1235/2449] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:20:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:20:37.765770 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c74c18-1add-47a7-8c46-9aedda15d906 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:20:37.777072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c74c18-1add-47a7-8c46-9aedda15d906 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:20:37.783020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3d5fb-751a-416f-bb37-87a88dcc7d3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:20:37.783604 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3d5fb-751a-416f-bb37-87a88dcc7d3c service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:20:37.783878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3d5fb-751a-416f-bb37-87a88dcc7d3c service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:20:37.784146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3d5fb-751a-416f-bb37-87a88dcc7d3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:20:37.791797 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3d5fb-751a-416f-bb37-87a88dcc7d3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:20:37.793356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3d5fb-751a-416f-bb37-87a88dcc7d3c service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:20:37.819958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1215/2450] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:20:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:20:37.889426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da75d9c-d772-4e4d-a7ce-d1c4cfd61614 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:20:37.898937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da75d9c-d772-4e4d-a7ce-d1c4cfd61614 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:20:37.902506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b4721-1713-46fa-884e-1a92fae73892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:20:37.903097 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b4721-1713-46fa-884e-1a92fae73892 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:20:37.903368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b4721-1713-46fa-884e-1a92fae73892 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:20:37.903631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b4721-1713-46fa-884e-1a92fae73892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:20:37.914442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b4721-1713-46fa-884e-1a92fae73892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:20:37.915858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b4721-1713-46fa-884e-1a92fae73892 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:20:37.927973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1236/2451] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:20:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:20:47.817388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=35 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:20:47.923184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=24 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:22:24.519930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3d5fb-751a-416f-bb37-87a88dcc7d3c service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:22:24.539757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3d5fb-751a-416f-bb37-87a88dcc7d3c service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:22:24.543452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a01ccb-4cdd-4b00-aab8-5f8bce87c0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:22:24.544052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a01ccb-4cdd-4b00-aab8-5f8bce87c0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:22:24.544336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a01ccb-4cdd-4b00-aab8-5f8bce87c0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:22:24.544599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a01ccb-4cdd-4b00-aab8-5f8bce87c0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:22:24.552855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a01ccb-4cdd-4b00-aab8-5f8bce87c0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:22:24.554345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a01ccb-4cdd-4b00-aab8-5f8bce87c0e5 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:22:24.565411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1216/2452] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:22:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:22:34.558876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=10 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:22:37.359387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b4721-1713-46fa-884e-1a92fae73892 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:22:37.379066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b4721-1713-46fa-884e-1a92fae73892 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:22:37.382805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590575ac-e16c-40a8-89e9-a5b7a6bee143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:22:37.383583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590575ac-e16c-40a8-89e9-a5b7a6bee143 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:22:37.383935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590575ac-e16c-40a8-89e9-a5b7a6bee143 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:22:37.384109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590575ac-e16c-40a8-89e9-a5b7a6bee143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:22:37.385913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-590575ac-e16c-40a8-89e9-a5b7a6bee143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:22:37.473390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590575ac-e16c-40a8-89e9-a5b7a6bee143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:22:37.475021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590575ac-e16c-40a8-89e9-a5b7a6bee143 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:22:37.493535 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1237/2453] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:22:38.544559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a01ccb-4cdd-4b00-aab8-5f8bce87c0e5 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:22:38.557582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a01ccb-4cdd-4b00-aab8-5f8bce87c0e5 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:22:38.563165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bfa8c-e91b-40c5-82a4-4d4b2ea33d86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:22:38.563755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bfa8c-e91b-40c5-82a4-4d4b2ea33d86 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:22:38.564032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bfa8c-e91b-40c5-82a4-4d4b2ea33d86 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:22:38.564298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bfa8c-e91b-40c5-82a4-4d4b2ea33d86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:22:38.572176 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bfa8c-e91b-40c5-82a4-4d4b2ea33d86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:22:38.573747 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bfa8c-e91b-40c5-82a4-4d4b2ea33d86 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:22:38.603491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1217/2454] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:22:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:22:41.203599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590575ac-e16c-40a8-89e9-a5b7a6bee143 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:22:41.231746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590575ac-e16c-40a8-89e9-a5b7a6bee143 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:22:41.238045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e423-7dd8-4888-89b9-e45e6c84c6b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:22:41.238733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e423-7dd8-4888-89b9-e45e6c84c6b8 service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:22:41.239084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e423-7dd8-4888-89b9-e45e6c84c6b8 service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:22:41.239916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e423-7dd8-4888-89b9-e45e6c84c6b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:22:41.241780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7da2e423-7dd8-4888-89b9-e45e6c84c6b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:22:41.254224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2e423-7dd8-4888-89b9-e45e6c84c6b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:22:41.255887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2e423-7dd8-4888-89b9-e45e6c84c6b8 service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:22:41.296658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1238/2455] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:22:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:22:44.637220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bfa8c-e91b-40c5-82a4-4d4b2ea33d86 service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:22:44.638690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bf3bfa8c-e91b-40c5-82a4-4d4b2ea33d86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:22:44.652864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bfa8c-e91b-40c5-82a4-4d4b2ea33d86 service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:22:44.657903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:22:44.660457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:22:44.660457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:22:44.660457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] PATH_INFO: `/v3/users/9563998855204d3881af170730f04a34` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:22:44.660457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9563998855204d3881af170730f04a34)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:22:44.662092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:22:44.855620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9563998855204d3881af170730f04a34'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:44.931822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9563998855204d3881af170730f04a34'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:44.944195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9563998855204d3881af170730f04a34'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:44.951305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9563998855204d3881af170730f04a34'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:44.951942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9563998855204d3881af170730f04a34'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:44.957888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1218/2456] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:22:44 2022] DELETE /identity/v3/users/9563998855204d3881af170730f04a34 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:22:45.219714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e423-7dd8-4888-89b9-e45e6c84c6b8 service placement] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:22:45.231283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e423-7dd8-4888-89b9-e45e6c84c6b8 service placement] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:22:45.288805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:22:45.289378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:22:45.289634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:22:45.289934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] PATH_INFO: `/v3/projects/42851a2730ad4d749d5e33d44ea607e6` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:22:45.291032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] RBAC: Authorizing `identity:delete_project(project_id=42851a2730ad4d749d5e33d44ea607e6)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:22:45.292920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:22:45.393386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:22:45.396228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42851a2730ad4d749d5e33d44ea607e6', 'request_id': 'req-349936a4-3035-48f8-a6c1-611aeaa1b28f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.402318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42851a2730ad4d749d5e33d44ea607e6', 'request_id': 'req-349936a4-3035-48f8-a6c1-611aeaa1b28f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.492248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42851a2730ad4d749d5e33d44ea607e6', 'request_id': 'req-349936a4-3035-48f8-a6c1-611aeaa1b28f'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.503632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] The token cache is being invalidate because project 42851a2730ad4d749d5e33d44ea607e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:22:45.503916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.504545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.506389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1239/2457] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:22:45 2022] DELETE /identity/v3/projects/42851a2730ad4d749d5e33d44ea607e6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:22:45.510708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:22:45.512296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:22:45.526801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c875-4d5f-43f9-b372-8b91088d3116 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:22:45.582817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:22:45.583351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:22:45.583628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:22:45.583888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] PATH_INFO: `/v3/users/ca46d8f22e9749a39ff5f27986c762c9` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:22:45.597653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca46d8f22e9749a39ff5f27986c762c9)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:22:45.599628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:22:45.737010 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:22:45.746039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca46d8f22e9749a39ff5f27986c762c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.840149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca46d8f22e9749a39ff5f27986c762c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.852608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca46d8f22e9749a39ff5f27986c762c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.860235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca46d8f22e9749a39ff5f27986c762c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.862627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca46d8f22e9749a39ff5f27986c762c9'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:45.867141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1219/2458] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:22:45 2022] DELETE /identity/v3/users/ca46d8f22e9749a39ff5f27986c762c9 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:22:46.110439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:22:46.111898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:22:46.125215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349936a4-3035-48f8-a6c1-611aeaa1b28f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:22:46.184718 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:22:46.185351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:22:46.185628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:22:46.185936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] PATH_INFO: `/v3/projects/7b0fb315a2a94894b87088aa0b8472c3` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:22:46.196197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0fb315a2a94894b87088aa0b8472c3)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:22:46.198137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:22:46.315370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0fb315a2a94894b87088aa0b8472c3', 'request_id': 'req-9693c576-e3e6-49ce-b96d-38ee67594a73'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:46.322207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0fb315a2a94894b87088aa0b8472c3', 'request_id': 'req-9693c576-e3e6-49ce-b96d-38ee67594a73'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:46.371851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0fb315a2a94894b87088aa0b8472c3', 'request_id': 'req-9693c576-e3e6-49ce-b96d-38ee67594a73'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:46.382160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] The token cache is being invalidate because project 7b0fb315a2a94894b87088aa0b8472c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:22:46.385366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:46.385366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:22:46.385366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1240/2459] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:22:46 2022] DELETE /identity/v3/projects/7b0fb315a2a94894b87088aa0b8472c3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:22:55.909225 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:22:56.379152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:DELETE=11,keystone:SELECT=62,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:24:21.986838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:24:21.988322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:24:22.060152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d8309-057e-4952-8651-01af67cc52cb admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:24:22.125221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:24:22.125856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:24:22.126141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:24:22.126411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:24:22.128462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:24:22.144905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:24:22.145360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:24:22.156375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1220/2460] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:24:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:24:32.152669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=58 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:24:34.790296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:24:34.791633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:24:34.869444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693c576-e3e6-49ce-b96d-38ee67594a73 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:24:34.873094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:24:34.873701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:24:34.874037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:24:34.874312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:24:34.876065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:24:34.950031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:24:34.951952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:24:34.970906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1241/2461] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:24:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:24:44.968074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=65 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:25:37.296911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:25:37.298436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:37.330308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:25:37.331732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:37.353576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265cd4a-cbce-478b-b430-292293b9b458 service placement] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:25:37.357780 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:25:37.358402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:37.358681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:37.359349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:37.361208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:37.409012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29711921-9bf5-487e-a401-2e5b86d591bf service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:25:37.414993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:25:37.415602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:37.415880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:37.416154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:37.417936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:37.432095 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:25:37.433543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:25:37.444367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1221/2462] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:25:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:25:37.477838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:25:37.479453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:25:37.489354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1242/2463] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:25:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:25:37.540082 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:25:37.541528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:37.552880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04315914-f821-4ae4-ac22-90bc839d9d9b service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:25:37.556668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321246c2-7171-44c5-a914-5a5f67c93b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:25:37.557324 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321246c2-7171-44c5-a914-5a5f67c93b43 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:37.557599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321246c2-7171-44c5-a914-5a5f67c93b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:37.557894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321246c2-7171-44c5-a914-5a5f67c93b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:37.559682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-321246c2-7171-44c5-a914-5a5f67c93b43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:37.573407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321246c2-7171-44c5-a914-5a5f67c93b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:25:37.574847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321246c2-7171-44c5-a914-5a5f67c93b43 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:25:37.585586 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1222/2464] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:25:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:25:39.783746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:25:39.784529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:39.797074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938b307-99fe-4c1a-b6a3-ec19101b75d1 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:25:39.800693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527c90e-6929-46b8-80bd-18ad83089b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:25:39.801432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527c90e-6929-46b8-80bd-18ad83089b71 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:39.801708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527c90e-6929-46b8-80bd-18ad83089b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:39.802014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527c90e-6929-46b8-80bd-18ad83089b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:39.806230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e527c90e-6929-46b8-80bd-18ad83089b71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application [None req-e527c90e-6929-46b8-80bd-18ad83089b71 service cinder] Could not find project: 42851a2730ad4d749d5e33d44ea607e6.: keystone.exception.ProjectNotFound: Could not find project: 42851a2730ad4d749d5e33d44ea607e6. Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 08 17:25:39.820044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:25:39.821188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 17:25:39.822382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application with Lock( Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._enter() Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.creator() Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 17:25:39.826394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 17:25:39.827705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 42851a2730ad4d749d5e33d44ea607e6. Aug 08 17:25:39.827705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: ERROR keystone.server.flask.application Aug 08 17:25:39.827705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1243/2465] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:25:39 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 08 17:25:39.857579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321246c2-7171-44c5-a914-5a5f67c93b43 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:25:39.867003 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321246c2-7171-44c5-a914-5a5f67c93b43 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:25:39.870700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b74dd-7fef-4260-8c22-68bc8c247bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:25:39.871794 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b74dd-7fef-4260-8c22-68bc8c247bbd service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:39.872066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b74dd-7fef-4260-8c22-68bc8c247bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:39.872401 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b74dd-7fef-4260-8c22-68bc8c247bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:39.874148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e88b74dd-7fef-4260-8c22-68bc8c247bbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:39.944867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b74dd-7fef-4260-8c22-68bc8c247bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:25:39.946331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b74dd-7fef-4260-8c22-68bc8c247bbd service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:25:39.955852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1223/2466] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:25:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:25:44.414132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e83fea-a6b3-4373-8674-eb69b9176aec None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:44.414635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e83fea-a6b3-4373-8674-eb69b9176aec None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:44.414736 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e83fea-a6b3-4373-8674-eb69b9176aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:44.452802 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-15e83fea-a6b3-4373-8674-eb69b9176aec None None] MFA Rules not processed for user `60dc4976d686449f80d3bfad387a2406`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:25:44.454020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-15e83fea-a6b3-4373-8674-eb69b9176aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:44.460722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1244/2467] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:25:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:25:44.776243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b74dd-7fef-4260-8c22-68bc8c247bbd service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:25:44.786331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b74dd-7fef-4260-8c22-68bc8c247bbd service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:25:44.789919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:25:44.790510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:44.790789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:44.791086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:44.792837 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:44.802181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:25:44.803579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:25:44.811293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1224/2468] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:25:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:25:45.262240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e83fea-a6b3-4373-8674-eb69b9176aec None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:25:45.263620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-15e83fea-a6b3-4373-8674-eb69b9176aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:45.278725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e83fea-a6b3-4373-8674-eb69b9176aec None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:25:45.283673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6254b-6a4e-4f9e-a0b1-fb69e173cd91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:25:45.284275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6254b-6a4e-4f9e-a0b1-fb69e173cd91 service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:45.284550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6254b-6a4e-4f9e-a0b1-fb69e173cd91 service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:45.284815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6254b-6a4e-4f9e-a0b1-fb69e173cd91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:45.286543 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-15b6254b-6a4e-4f9e-a0b1-fb69e173cd91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:45.299332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6254b-6a4e-4f9e-a0b1-fb69e173cd91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:25:45.300905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6254b-6a4e-4f9e-a0b1-fb69e173cd91 service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:25:45.311702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1245/2469] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:25:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:25:49.143905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:25:49.145565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:49.149137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:25:49.175492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f2b43-7b97-46b6-b2b3-533f6ac313f0 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:25:49.181019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3263337-a263-4067-8448-a6f7b1b255ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:25:49.182425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3263337-a263-4067-8448-a6f7b1b255ca service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:49.182694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3263337-a263-4067-8448-a6f7b1b255ca service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:49.184097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3263337-a263-4067-8448-a6f7b1b255ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:49.185975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c3263337-a263-4067-8448-a6f7b1b255ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:49.198458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3263337-a263-4067-8448-a6f7b1b255ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:25:49.200045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3263337-a263-4067-8448-a6f7b1b255ca service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:25:49.211160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1225/2470] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:25:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:25:55.306700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=77 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:25:56.125218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6254b-6a4e-4f9e-a0b1-fb69e173cd91 service placement] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:25:56.137625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6254b-6a4e-4f9e-a0b1-fb69e173cd91 service placement] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:25:56.143036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4098836-98a1-4351-9c80-c3717736931d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:25:56.147208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4098836-98a1-4351-9c80-c3717736931d service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:25:56.147208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4098836-98a1-4351-9c80-c3717736931d service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:25:56.147208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4098836-98a1-4351-9c80-c3717736931d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:25:56.147208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c4098836-98a1-4351-9c80-c3717736931d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:25:56.158909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4098836-98a1-4351-9c80-c3717736931d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:25:56.160482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4098836-98a1-4351-9c80-c3717736931d service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:25:56.170962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1246/2471] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:25:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:25:59.222806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=10 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:26:00.460863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3263337-a263-4067-8448-a6f7b1b255ca service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:26:00.472410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3263337-a263-4067-8448-a6f7b1b255ca service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:26:00.477812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dd259-1374-48bb-a369-9eba08531330 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:26:00.479890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd259-1374-48bb-a369-9eba08531330 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:26:00.480162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd259-1374-48bb-a369-9eba08531330 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:26:00.480427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dd259-1374-48bb-a369-9eba08531330 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:26:00.482774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-000dd259-1374-48bb-a369-9eba08531330 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:26:00.495178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dd259-1374-48bb-a369-9eba08531330 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:26:00.496900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dd259-1374-48bb-a369-9eba08531330 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:26:00.508605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1226/2472] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:26:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:26:06.166452 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=6 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:26:10.504243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=6 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:27:39.496076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4098836-98a1-4351-9c80-c3717736931d service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:27:39.507466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4098836-98a1-4351-9c80-c3717736931d service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:27:39.513575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:27:39.514244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:27:39.514525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:27:39.514794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:27:39.516558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:27:39.597299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:27:39.598695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:27:39.608331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1247/2473] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:27:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:27:49.603820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=33 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:28:32.634648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dd259-1374-48bb-a369-9eba08531330 service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:28:32.643829 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dd259-1374-48bb-a369-9eba08531330 service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:28:32.647355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26213860-cf22-4b20-bbef-53b3f00640d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:28:32.647932 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213860-cf22-4b20-bbef-53b3f00640d2 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:28:32.648202 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213860-cf22-4b20-bbef-53b3f00640d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:28:32.648465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213860-cf22-4b20-bbef-53b3f00640d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:28:32.650178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-26213860-cf22-4b20-bbef-53b3f00640d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:28:32.671559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26213860-cf22-4b20-bbef-53b3f00640d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:28:32.673012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26213860-cf22-4b20-bbef-53b3f00640d2 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:28:32.681829 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1227/2474] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:28:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:28:42.678155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=11 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:29:23.930170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:29:23.931660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:29:23.996235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55eed3-ba8e-4915-99de-405038d02f2d service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:29:24.001277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ffc7a-f712-43ab-a558-3024b431b4a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:29:24.001899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ffc7a-f712-43ab-a558-3024b431b4a5 service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:29:24.002178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ffc7a-f712-43ab-a558-3024b431b4a5 service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:29:24.002445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ffc7a-f712-43ab-a558-3024b431b4a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:29:24.011432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ffc7a-f712-43ab-a558-3024b431b4a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:29:24.012994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ffc7a-f712-43ab-a558-3024b431b4a5 service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:29:24.023315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1248/2475] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:29:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:29:34.018551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=28 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:29:34.505496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26213860-cf22-4b20-bbef-53b3f00640d2 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:29:34.518629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26213860-cf22-4b20-bbef-53b3f00640d2 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:29:34.524615 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257603fb-e8a2-4058-80d6-9d3a29434696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:29:34.525212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257603fb-e8a2-4058-80d6-9d3a29434696 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:29:34.525505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257603fb-e8a2-4058-80d6-9d3a29434696 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:29:34.525817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257603fb-e8a2-4058-80d6-9d3a29434696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:29:34.533727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257603fb-e8a2-4058-80d6-9d3a29434696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:29:34.535329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257603fb-e8a2-4058-80d6-9d3a29434696 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:29:34.545207 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1228/2476] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:29:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:29:44.540592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=6 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:30:38.330319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ffc7a-f712-43ab-a558-3024b431b4a5 service placement] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:30:38.349291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ffc7a-f712-43ab-a558-3024b431b4a5 service placement] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:30:38.353489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac15747-0b4e-40fc-9397-10fcfe85d9be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:30:38.354118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac15747-0b4e-40fc-9397-10fcfe85d9be service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:30:38.354382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac15747-0b4e-40fc-9397-10fcfe85d9be service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:30:38.354649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac15747-0b4e-40fc-9397-10fcfe85d9be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:30:38.371778 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac15747-0b4e-40fc-9397-10fcfe85d9be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:30:38.373186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac15747-0b4e-40fc-9397-10fcfe85d9be service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:30:38.393723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1249/2477] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:30:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:30:38.775884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bd6ea-ea5f-4cd8-ba5e-b857a7bafe18 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:30:38.776424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bd6ea-ea5f-4cd8-ba5e-b857a7bafe18 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:30:38.776424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bd6ea-ea5f-4cd8-ba5e-b857a7bafe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:30:38.825587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-710bd6ea-ea5f-4cd8-ba5e-b857a7bafe18 None None] MFA Rules not processed for user `5a7ed5f2559a449fbe0b30bfeabd90df`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:30:38.826760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-710bd6ea-ea5f-4cd8-ba5e-b857a7bafe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:30:38.833550 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1229/2478] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:30:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:30:39.563196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac15747-0b4e-40fc-9397-10fcfe85d9be service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:30:39.576813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac15747-0b4e-40fc-9397-10fcfe85d9be service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:30:39.589546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a9eb5-f706-480d-b229-7f450b17478f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:30:39.590216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a9eb5-f706-480d-b229-7f450b17478f service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:30:39.590496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a9eb5-f706-480d-b229-7f450b17478f service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:30:39.590762 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a9eb5-f706-480d-b229-7f450b17478f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:30:39.604476 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a9eb5-f706-480d-b229-7f450b17478f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:30:39.606111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a9eb5-f706-480d-b229-7f450b17478f service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:30:39.634652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1250/2479] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:30:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:30:48.822906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=15 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:30:49.631101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=31 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:31:40.854951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bd6ea-ea5f-4cd8-ba5e-b857a7bafe18 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:31:40.864905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bd6ea-ea5f-4cd8-ba5e-b857a7bafe18 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:31:40.869145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70913c-e86c-43d4-9d70-f8b3a12df52e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:31:40.869732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70913c-e86c-43d4-9d70-f8b3a12df52e service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:31:40.870041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70913c-e86c-43d4-9d70-f8b3a12df52e service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:31:40.870308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70913c-e86c-43d4-9d70-f8b3a12df52e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:31:40.878771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70913c-e86c-43d4-9d70-f8b3a12df52e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:31:40.887706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70913c-e86c-43d4-9d70-f8b3a12df52e service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:31:40.907697 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1230/2480] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:31:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:31:50.905268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=13 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:32:32.428614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a9eb5-f706-480d-b229-7f450b17478f service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:32:32.440844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a9eb5-f706-480d-b229-7f450b17478f service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:32:32.446482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065103-b211-4d59-87e0-8b92d7fe04c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:32:32.447088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065103-b211-4d59-87e0-8b92d7fe04c6 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:32:32.447368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065103-b211-4d59-87e0-8b92d7fe04c6 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:32:32.447635 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065103-b211-4d59-87e0-8b92d7fe04c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:32:32.457750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065103-b211-4d59-87e0-8b92d7fe04c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:32:32.459401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065103-b211-4d59-87e0-8b92d7fe04c6 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:32:32.470039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1251/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:32:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:32:37.762603 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3aa1e-1487-4d39-8bd0-07ca0799f287 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:32:37.763134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3aa1e-1487-4d39-8bd0-07ca0799f287 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:32:37.763134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3aa1e-1487-4d39-8bd0-07ca0799f287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:32:37.782493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c2f3aa1e-1487-4d39-8bd0-07ca0799f287 None None] MFA Rules not processed for user `5a7ed5f2559a449fbe0b30bfeabd90df`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:32:37.787064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c2f3aa1e-1487-4d39-8bd0-07ca0799f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:32:37.790754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1231/2482] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:32:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:32:38.086125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065103-b211-4d59-87e0-8b92d7fe04c6 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:32:38.105763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065103-b211-4d59-87e0-8b92d7fe04c6 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:32:38.109590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2f36f-ea22-42ef-8248-1aa995651163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:32:38.110243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2f36f-ea22-42ef-8248-1aa995651163 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:32:38.110518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2f36f-ea22-42ef-8248-1aa995651163 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:32:38.110792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2f36f-ea22-42ef-8248-1aa995651163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:32:38.117744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2f36f-ea22-42ef-8248-1aa995651163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:32:38.119173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2f36f-ea22-42ef-8248-1aa995651163 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:32:38.127408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1252/2483] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:32:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:32:38.524434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3aa1e-1487-4d39-8bd0-07ca0799f287 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:32:38.536685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3aa1e-1487-4d39-8bd0-07ca0799f287 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:32:38.542130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6e8c7-d647-477f-b0c4-aab93281e95b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:32:38.542974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6e8c7-d647-477f-b0c4-aab93281e95b service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:32:38.543439 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6e8c7-d647-477f-b0c4-aab93281e95b service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:32:38.543892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6e8c7-d647-477f-b0c4-aab93281e95b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:32:38.554032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6e8c7-d647-477f-b0c4-aab93281e95b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:32:38.555805 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6e8c7-d647-477f-b0c4-aab93281e95b service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:32:38.567217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1232/2484] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:32:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:32:48.124036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=16 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:32:48.211246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2f36f-ea22-42ef-8248-1aa995651163 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:32:48.224143 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2f36f-ea22-42ef-8248-1aa995651163 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:32:48.230088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f85387-30ea-43ed-a202-b5983f89bdfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:32:48.230684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f85387-30ea-43ed-a202-b5983f89bdfa service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:32:48.230964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f85387-30ea-43ed-a202-b5983f89bdfa service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:32:48.231237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f85387-30ea-43ed-a202-b5983f89bdfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:32:48.240372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f85387-30ea-43ed-a202-b5983f89bdfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:32:48.241976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f85387-30ea-43ed-a202-b5983f89bdfa service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:32:48.252968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1253/2485] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:32:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:32:48.563344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=9 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:32:53.090713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6e8c7-d647-477f-b0c4-aab93281e95b service placement] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:32:53.103312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6e8c7-d647-477f-b0c4-aab93281e95b service placement] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:32:53.109153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3f470-6a1f-4170-985e-1bd931fe7f65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:32:53.109743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3f470-6a1f-4170-985e-1bd931fe7f65 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:32:53.110063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3f470-6a1f-4170-985e-1bd931fe7f65 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:32:53.110331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3f470-6a1f-4170-985e-1bd931fe7f65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:32:53.120077 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3f470-6a1f-4170-985e-1bd931fe7f65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:32:53.121679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3f470-6a1f-4170-985e-1bd931fe7f65 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:32:53.132081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1233/2486] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:32:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:32:58.248580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=6 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:33:03.127658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=6 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:33:41.021187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f85387-30ea-43ed-a202-b5983f89bdfa service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:33:41.030404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f85387-30ea-43ed-a202-b5983f89bdfa service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:33:41.033939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7ace8-f39b-43f9-887e-4c1b774a87f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:33:41.034528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7ace8-f39b-43f9-887e-4c1b774a87f7 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:33:41.034805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7ace8-f39b-43f9-887e-4c1b774a87f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:33:41.035075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7ace8-f39b-43f9-887e-4c1b774a87f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:33:41.043684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7ace8-f39b-43f9-887e-4c1b774a87f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:33:41.045089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7ace8-f39b-43f9-887e-4c1b774a87f7 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:33:41.072407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1254/2487] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:33:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:33:51.069848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=13 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:34:26.899519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3f470-6a1f-4170-985e-1bd931fe7f65 service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:26.964058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3f470-6a1f-4170-985e-1bd931fe7f65 service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:26.968991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e560f-d9ff-4124-860e-4504f87e8865 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:26.969572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e560f-d9ff-4124-860e-4504f87e8865 service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:26.969868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e560f-d9ff-4124-860e-4504f87e8865 service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:26.970133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e560f-d9ff-4124-860e-4504f87e8865 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:26.971923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1c3e560f-d9ff-4124-860e-4504f87e8865 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:34:26.983913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e560f-d9ff-4124-860e-4504f87e8865 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:26.985662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e560f-d9ff-4124-860e-4504f87e8865 service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:26.996503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1234/2488] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:34:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:34:36.992126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=32 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:34:37.327456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7ace8-f39b-43f9-887e-4c1b774a87f7 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:37.329059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cea7ace8-f39b-43f9-887e-4c1b774a87f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:34:37.409568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7ace8-f39b-43f9-887e-4c1b774a87f7 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:37.413180 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42732d-e46f-48cd-827b-efa82f54302b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:37.413776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42732d-e46f-48cd-827b-efa82f54302b service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:37.414081 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42732d-e46f-48cd-827b-efa82f54302b service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:37.414354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42732d-e46f-48cd-827b-efa82f54302b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:37.416187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-aa42732d-e46f-48cd-827b-efa82f54302b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:34:37.481624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42732d-e46f-48cd-827b-efa82f54302b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:37.483071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42732d-e46f-48cd-827b-efa82f54302b service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:37.500959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1255/2489] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:34:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:34:37.668658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e560f-d9ff-4124-860e-4504f87e8865 service placement] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:37.677733 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e560f-d9ff-4124-860e-4504f87e8865 service placement] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:37.681859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c69a6-6c6d-47f1-8670-bae936494c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:37.682433 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c69a6-6c6d-47f1-8670-bae936494c0a service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:37.682704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c69a6-6c6d-47f1-8670-bae936494c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:37.682968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c69a6-6c6d-47f1-8670-bae936494c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:37.684647 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-640c69a6-6c6d-47f1-8670-bae936494c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:34:37.765148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c69a6-6c6d-47f1-8670-bae936494c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:37.766519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c69a6-6c6d-47f1-8670-bae936494c0a service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:37.784800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1235/2490] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:34:39.641497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42732d-e46f-48cd-827b-efa82f54302b service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:39.651516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42732d-e46f-48cd-827b-efa82f54302b service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:39.655125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55586a-fd28-41aa-80d7-15cb27a1318e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:39.655730 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55586a-fd28-41aa-80d7-15cb27a1318e service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:39.656006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55586a-fd28-41aa-80d7-15cb27a1318e service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:39.656275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55586a-fd28-41aa-80d7-15cb27a1318e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:39.664068 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55586a-fd28-41aa-80d7-15cb27a1318e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:39.667421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55586a-fd28-41aa-80d7-15cb27a1318e service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:39.676505 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1256/2491] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:34:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:34:41.992423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c69a6-6c6d-47f1-8670-bae936494c0a service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:42.006828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c69a6-6c6d-47f1-8670-bae936494c0a service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:42.012587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af0dae-65a7-41be-aa09-df2b0c07295e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:42.013181 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af0dae-65a7-41be-aa09-df2b0c07295e service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:42.013454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af0dae-65a7-41be-aa09-df2b0c07295e service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:42.013718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af0dae-65a7-41be-aa09-df2b0c07295e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:42.015456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-65af0dae-65a7-41be-aa09-df2b0c07295e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:34:42.026917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af0dae-65a7-41be-aa09-df2b0c07295e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:42.028538 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af0dae-65a7-41be-aa09-df2b0c07295e service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:42.038639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1236/2492] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:34:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:34:42.104584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55586a-fd28-41aa-80d7-15cb27a1318e service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:42.106151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ff55586a-fd28-41aa-80d7-15cb27a1318e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:34:42.124263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55586a-fd28-41aa-80d7-15cb27a1318e service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:42.129067 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:42.129741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:42.130093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:42.130367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] PATH_INFO: `/v3/users/60dc4976d686449f80d3bfad387a2406` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:42.131489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] RBAC: Authorizing `identity:delete_user(user_id=60dc4976d686449f80d3bfad387a2406)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:42.133449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:42.328275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60dc4976d686449f80d3bfad387a2406'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.411791 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60dc4976d686449f80d3bfad387a2406'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.417136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:34:42.420126 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60dc4976d686449f80d3bfad387a2406'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.432623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60dc4976d686449f80d3bfad387a2406'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.433288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-db66bdfd-659c-40d1-ad50-d1cb10d8e8df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60dc4976d686449f80d3bfad387a2406'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.439385 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1257/2493] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:34:42 2022] DELETE /identity/v3/users/60dc4976d686449f80d3bfad387a2406 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:34:42.688002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af0dae-65a7-41be-aa09-df2b0c07295e service placement] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:42.699087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af0dae-65a7-41be-aa09-df2b0c07295e service placement] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:42.763517 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:42.764167 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:42.764443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:42.764713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] PATH_INFO: `/v3/projects/3b8ed5fb092140a1a4e9ae005f3c8973` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:42.765993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b8ed5fb092140a1a4e9ae005f3c8973)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:42.767921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:42.873466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b8ed5fb092140a1a4e9ae005f3c8973', 'request_id': 'req-07951ae3-a347-4ad5-be96-8b96455aa2ad'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.879856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b8ed5fb092140a1a4e9ae005f3c8973', 'request_id': 'req-07951ae3-a347-4ad5-be96-8b96455aa2ad'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.881493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:34:42.954666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b8ed5fb092140a1a4e9ae005f3c8973', 'request_id': 'req-07951ae3-a347-4ad5-be96-8b96455aa2ad'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.965630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] The token cache is being invalidate because project 3b8ed5fb092140a1a4e9ae005f3c8973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:34:42.965976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.966614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-07951ae3-a347-4ad5-be96-8b96455aa2ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:34:42.968063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1237/2494] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:34:42 2022] DELETE /identity/v3/projects/3b8ed5fb092140a1a4e9ae005f3c8973 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:34:42.974578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a9f5e-3518-4377-b72e-8b23d3e13972 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:42.980351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a9f5e-3518-4377-b72e-8b23d3e13972 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:42.980351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a9f5e-3518-4377-b72e-8b23d3e13972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:43.033639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d13a9f5e-3518-4377-b72e-8b23d3e13972 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:34:43.034923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d13a9f5e-3518-4377-b72e-8b23d3e13972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:34:43.096495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1258/2495] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:34:43.105429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9075ac-3c5a-4fe1-9efd-0a45e9787821 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:43.105955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9075ac-3c5a-4fe1-9efd-0a45e9787821 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:43.106073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9075ac-3c5a-4fe1-9efd-0a45e9787821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:43.125591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0d9075ac-3c5a-4fe1-9efd-0a45e9787821 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:34:43.126810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0d9075ac-3c5a-4fe1-9efd-0a45e9787821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:34:43.135267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1238/2496] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:34:43.139457 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a9f5e-3518-4377-b72e-8b23d3e13972 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:43.149166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a9f5e-3518-4377-b72e-8b23d3e13972 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:43.157358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d388ba3-1376-4050-afe9-eb1c744f295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:43.158071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388ba3-1376-4050-afe9-eb1c744f295b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:43.158406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388ba3-1376-4050-afe9-eb1c744f295b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:43.158677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388ba3-1376-4050-afe9-eb1c744f295b admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:43.159329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d388ba3-1376-4050-afe9-eb1c744f295b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:34:43.159629 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d388ba3-1376-4050-afe9-eb1c744f295b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:43.161623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d388ba3-1376-4050-afe9-eb1c744f295b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:43.173742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1259/2497] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:34:43.180694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9075ac-3c5a-4fe1-9efd-0a45e9787821 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:43.189807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9075ac-3c5a-4fe1-9efd-0a45e9787821 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:43.194497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cacfc-4290-47dc-b38e-d54e9ef4e08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:43.195076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cacfc-4290-47dc-b38e-d54e9ef4e08b admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:43.195345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cacfc-4290-47dc-b38e-d54e9ef4e08b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:43.195605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cacfc-4290-47dc-b38e-d54e9ef4e08b admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:43.196739 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cacfc-4290-47dc-b38e-d54e9ef4e08b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:43.198636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cacfc-4290-47dc-b38e-d54e9ef4e08b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:43.281515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1239/2498] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:34:43 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:34:43.285776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d388ba3-1376-4050-afe9-eb1c744f295b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:43.295720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d388ba3-1376-4050-afe9-eb1c744f295b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:43.350906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5d5a-d573-4507-9d74-07df86d60059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:43.351504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5d5a-d573-4507-9d74-07df86d60059 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:43.351783 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5d5a-d573-4507-9d74-07df86d60059 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:43.352307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5d5a-d573-4507-9d74-07df86d60059 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:43.352922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff5d5a-d573-4507-9d74-07df86d60059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:43.354926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff5d5a-d573-4507-9d74-07df86d60059 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:43.457437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1260/2499] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:34:43 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:34:43.461505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cacfc-4290-47dc-b38e-d54e9ef4e08b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:43.471832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cacfc-4290-47dc-b38e-d54e9ef4e08b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:43.476723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561a242-08a7-4919-ade4-b59ac5aa8dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:43.477596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561a242-08a7-4919-ade4-b59ac5aa8dda admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:43.479360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561a242-08a7-4919-ade4-b59ac5aa8dda admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:43.479360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561a242-08a7-4919-ade4-b59ac5aa8dda admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:43.479360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561a242-08a7-4919-ade4-b59ac5aa8dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:43.480390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561a242-08a7-4919-ade4-b59ac5aa8dda admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:43.490177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1240/2500] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:34:43.494700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5d5a-d573-4507-9d74-07df86d60059 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:43.504969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5d5a-d573-4507-9d74-07df86d60059 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:43.510125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67f216-a48c-456a-89cb-18854195d3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:43.510710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67f216-a48c-456a-89cb-18854195d3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:43.510983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67f216-a48c-456a-89cb-18854195d3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:43.511275 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67f216-a48c-456a-89cb-18854195d3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:43.511907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67f216-a48c-456a-89cb-18854195d3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:43.513503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67f216-a48c-456a-89cb-18854195d3b8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:43.523937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1261/2501] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:34:43.527952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561a242-08a7-4919-ade4-b59ac5aa8dda admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:34:43.537725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561a242-08a7-4919-ade4-b59ac5aa8dda admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:34:43.542228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cf399-2f0a-4e6b-af64-21c75c5a3911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:34:43.542856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cf399-2f0a-4e6b-af64-21c75c5a3911 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:43.543134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cf399-2f0a-4e6b-af64-21c75c5a3911 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:43.543407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cf399-2f0a-4e6b-af64-21c75c5a3911 admin admin] PATH_INFO: `/v3/projects/d2406a0800994a1ab7737d513273fcae/users/4f561e2ff8854f77bbbd514ad1af2a74/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:43.557921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cf399-2f0a-4e6b-af64-21c75c5a3911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2406a0800994a1ab7737d513273fcae, user_id=4f561e2ff8854f77bbbd514ad1af2a74, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:34:43.560565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cf399-2f0a-4e6b-af64-21c75c5a3911 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:34:43.646023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1241/2502] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:34:43 2022] PUT /identity/v3/projects/d2406a0800994a1ab7737d513273fcae/users/4f561e2ff8854f77bbbd514ad1af2a74/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:34:43.663051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453945a8-937c-4f80-884c-e32602cc8e21 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:34:43.663335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453945a8-937c-4f80-884c-e32602cc8e21 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:34:43.663707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453945a8-937c-4f80-884c-e32602cc8e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:34:43.685578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:34:43.692648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-453945a8-937c-4f80-884c-e32602cc8e21 None None] MFA Rules not processed for user `4f561e2ff8854f77bbbd514ad1af2a74`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:34:43.693884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-453945a8-937c-4f80-884c-e32602cc8e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:34:43.757777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1262/2503] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:34:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:34:53.641989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=35 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:34:53.776743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=27 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:35:38.152525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cf399-2f0a-4e6b-af64-21c75c5a3911 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:35:38.154305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a67cf399-2f0a-4e6b-af64-21c75c5a3911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:35:38.208346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cf399-2f0a-4e6b-af64-21c75c5a3911 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:35:38.212319 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:35:38.212904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:35:38.213177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:35:38.213440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:35:38.215493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:35:38.284205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:35:38.285621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:35:38.296835 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1242/2504] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:35:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:35:40.594706 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453945a8-937c-4f80-884c-e32602cc8e21 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:35:40.596280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-453945a8-937c-4f80-884c-e32602cc8e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:35:40.612186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453945a8-937c-4f80-884c-e32602cc8e21 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:35:40.617485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d98934-9e5f-4f87-8195-8513f4032255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:35:40.618110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d98934-9e5f-4f87-8195-8513f4032255 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:35:40.618390 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d98934-9e5f-4f87-8195-8513f4032255 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:35:40.618655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d98934-9e5f-4f87-8195-8513f4032255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:35:40.620317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d1d98934-9e5f-4f87-8195-8513f4032255 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:35:40.689622 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d98934-9e5f-4f87-8195-8513f4032255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:35:40.691259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d98934-9e5f-4f87-8195-8513f4032255 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:35:40.700950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1263/2505] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:35:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:35:41.729377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:35:41.730776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:35:41.784643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0807a-3096-4c5e-8ce3-0082929aa017 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:35:41.788824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd477b-d797-40b5-8b9f-508dea21140a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:35:41.789422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd477b-d797-40b5-8b9f-508dea21140a service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:35:41.789693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd477b-d797-40b5-8b9f-508dea21140a service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:35:41.789985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd477b-d797-40b5-8b9f-508dea21140a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:35:41.798013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd477b-d797-40b5-8b9f-508dea21140a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:35:41.799415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd477b-d797-40b5-8b9f-508dea21140a service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:35:41.807930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1243/2506] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:35:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:35:50.697268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=33 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:35:51.804084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=80 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:46.761673 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d98934-9e5f-4f87-8195-8513f4032255 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:46.823870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d98934-9e5f-4f87-8195-8513f4032255 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:46.828280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:46.831297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:46.831297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:46.831297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] PATH_INFO: `/v3/users/4f561e2ff8854f77bbbd514ad1af2a74` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:46.831297 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f561e2ff8854f77bbbd514ad1af2a74)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:46.833197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:47.049536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f561e2ff8854f77bbbd514ad1af2a74'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:47.160836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f561e2ff8854f77bbbd514ad1af2a74'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:47.168555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f561e2ff8854f77bbbd514ad1af2a74'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:47.180954 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f561e2ff8854f77bbbd514ad1af2a74'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:47.186259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f561e2ff8854f77bbbd514ad1af2a74'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:47.189423 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1264/2507] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:36:46 2022] DELETE /identity/v3/users/4f561e2ff8854f77bbbd514ad1af2a74 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:47.194021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db5bda-7f76-42f1-973b-b2d257eb55e7 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:47.194547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db5bda-7f76-42f1-973b-b2d257eb55e7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:47.194547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db5bda-7f76-42f1-973b-b2d257eb55e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:47.214628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-80db5bda-7f76-42f1-973b-b2d257eb55e7 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:47.215957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-80db5bda-7f76-42f1-973b-b2d257eb55e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:47.290773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1244/2508] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:47.300006 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:47.301602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:47.370046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2de429-c742-4495-b45c-ddd01f27e4f2 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:47.375259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f3c23-36bc-4d25-9683-5cae7911a14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:47.375885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f3c23-36bc-4d25-9683-5cae7911a14f service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:47.376792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f3c23-36bc-4d25-9683-5cae7911a14f service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:47.377036 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f3c23-36bc-4d25-9683-5cae7911a14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:47.389096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f3c23-36bc-4d25-9683-5cae7911a14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:47.391676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f3c23-36bc-4d25-9683-5cae7911a14f service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:47.403744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1265/2509] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:36:47.791468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db5bda-7f76-42f1-973b-b2d257eb55e7 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:47.791468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-80db5bda-7f76-42f1-973b-b2d257eb55e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:47.803138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db5bda-7f76-42f1-973b-b2d257eb55e7 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:47.808080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:47.808707 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:47.809239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:47.809513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] PATH_INFO: `/v3/projects/d2406a0800994a1ab7737d513273fcae` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:47.810792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2406a0800994a1ab7737d513273fcae)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:47.812701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:47.929801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2406a0800994a1ab7737d513273fcae', 'request_id': 'req-fb2d37ad-07b0-4de0-8789-5572161a7e6e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:47.935805 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2406a0800994a1ab7737d513273fcae', 'request_id': 'req-fb2d37ad-07b0-4de0-8789-5572161a7e6e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:47.998269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2406a0800994a1ab7737d513273fcae', 'request_id': 'req-fb2d37ad-07b0-4de0-8789-5572161a7e6e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:48.009256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] The token cache is being invalidate because project d2406a0800994a1ab7737d513273fcae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:36:48.012901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:48.012901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-fb2d37ad-07b0-4de0-8789-5572161a7e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:48.012901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1245/2510] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:36:47 2022] DELETE /identity/v3/projects/d2406a0800994a1ab7737d513273fcae => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:48.016760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f137c-514b-4804-ac44-bbf91dbe6a83 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.017245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f137c-514b-4804-ac44-bbf91dbe6a83 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.017367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f137c-514b-4804-ac44-bbf91dbe6a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.037856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-cf1f137c-514b-4804-ac44-bbf91dbe6a83 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:48.039044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cf1f137c-514b-4804-ac44-bbf91dbe6a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:48.064780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:48.100330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1266/2511] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:36:48.108000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ce40b-b817-4d3f-8e29-f60a0403e069 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.108285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ce40b-b817-4d3f-8e29-f60a0403e069 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.108590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ce40b-b817-4d3f-8e29-f60a0403e069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.128599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-8a3ce40b-b817-4d3f-8e29-f60a0403e069 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:48.129838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8a3ce40b-b817-4d3f-8e29-f60a0403e069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:48.138147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1246/2512] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:48.142495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f137c-514b-4804-ac44-bbf91dbe6a83 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:48.153599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f137c-514b-4804-ac44-bbf91dbe6a83 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:48.158367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10747b2-e40c-4589-b76e-3359945f321e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:48.158968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10747b2-e40c-4589-b76e-3359945f321e admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.159248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10747b2-e40c-4589-b76e-3359945f321e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.159520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10747b2-e40c-4589-b76e-3359945f321e admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.160170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10747b2-e40c-4589-b76e-3359945f321e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:36:48.160472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10747b2-e40c-4589-b76e-3359945f321e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:48.162164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10747b2-e40c-4589-b76e-3359945f321e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:48.177221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1267/2513] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:36:48.178470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ce40b-b817-4d3f-8e29-f60a0403e069 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:48.188197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ce40b-b817-4d3f-8e29-f60a0403e069 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:48.194810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65eb5ad-bc0f-449d-bbdb-c1415ff9988b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:48.195419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65eb5ad-bc0f-449d-bbdb-c1415ff9988b admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.195674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65eb5ad-bc0f-449d-bbdb-c1415ff9988b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.195938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65eb5ad-bc0f-449d-bbdb-c1415ff9988b admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.196578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65eb5ad-bc0f-449d-bbdb-c1415ff9988b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:48.198760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65eb5ad-bc0f-449d-bbdb-c1415ff9988b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:48.298084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1247/2514] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:36:48 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:36:48.307962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10747b2-e40c-4589-b76e-3359945f321e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:48.326129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10747b2-e40c-4589-b76e-3359945f321e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:48.385614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697334a-e76d-4d1a-8fb9-4cb4c8d3ca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:48.386268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697334a-e76d-4d1a-8fb9-4cb4c8d3ca25 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.386727 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697334a-e76d-4d1a-8fb9-4cb4c8d3ca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.387014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697334a-e76d-4d1a-8fb9-4cb4c8d3ca25 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.387633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697334a-e76d-4d1a-8fb9-4cb4c8d3ca25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:48.389651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697334a-e76d-4d1a-8fb9-4cb4c8d3ca25 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:48.479145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1268/2515] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:36:48 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:36:48.480344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65eb5ad-bc0f-449d-bbdb-c1415ff9988b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:48.491320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65eb5ad-bc0f-449d-bbdb-c1415ff9988b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:48.496204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086a275-081e-4f24-89ce-ca7f1d0540c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:48.496851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086a275-081e-4f24-89ce-ca7f1d0540c9 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.497130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086a275-081e-4f24-89ce-ca7f1d0540c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.497436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086a275-081e-4f24-89ce-ca7f1d0540c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.498109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086a275-081e-4f24-89ce-ca7f1d0540c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:48.499701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086a275-081e-4f24-89ce-ca7f1d0540c9 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:48.510189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1248/2516] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:36:48.514368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697334a-e76d-4d1a-8fb9-4cb4c8d3ca25 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:48.525237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697334a-e76d-4d1a-8fb9-4cb4c8d3ca25 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:48.530038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933c2b2-f028-4fcb-b3e1-dfaa8780330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:48.530625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c2b2-f028-4fcb-b3e1-dfaa8780330b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.530903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c2b2-f028-4fcb-b3e1-dfaa8780330b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.531181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c2b2-f028-4fcb-b3e1-dfaa8780330b admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.531799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933c2b2-f028-4fcb-b3e1-dfaa8780330b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:48.533363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933c2b2-f028-4fcb-b3e1-dfaa8780330b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:48.543787 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1269/2517] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:36:48.547866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086a275-081e-4f24-89ce-ca7f1d0540c9 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:48.558665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086a275-081e-4f24-89ce-ca7f1d0540c9 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:48.564164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbc4e4-0fce-481e-8227-d5c5be4036ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:48.566093 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbc4e4-0fce-481e-8227-d5c5be4036ff admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.566371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbc4e4-0fce-481e-8227-d5c5be4036ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.566642 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbc4e4-0fce-481e-8227-d5c5be4036ff admin admin] PATH_INFO: `/v3/projects/70edc98b3178483b9352119793b35932/users/1636a19dc30141559c6039addaeccb37/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.581607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbc4e4-0fce-481e-8227-d5c5be4036ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=70edc98b3178483b9352119793b35932, user_id=1636a19dc30141559c6039addaeccb37, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:48.584408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbc4e4-0fce-481e-8227-d5c5be4036ff admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:48.681841 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1249/2518] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:36:48 2022] PUT /identity/v3/projects/70edc98b3178483b9352119793b35932/users/1636a19dc30141559c6039addaeccb37/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:48.693971 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e5118-d8f8-4416-bb5d-b491e42cb5df None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.694330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e5118-d8f8-4416-bb5d-b491e42cb5df None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.694610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e5118-d8f8-4416-bb5d-b491e42cb5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.731814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-5d8e5118-d8f8-4416-bb5d-b491e42cb5df None None] MFA Rules not processed for user `1636a19dc30141559c6039addaeccb37`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:48.731814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5d8e5118-d8f8-4416-bb5d-b491e42cb5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:48.800512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1270/2519] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:48.805096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbc4e4-0fce-481e-8227-d5c5be4036ff admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:48.826521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:48.868696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbc4e4-0fce-481e-8227-d5c5be4036ff admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:48.873374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ef9cf-1491-45e8-8c01-4a302c00a260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:48.874051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ef9cf-1491-45e8-8c01-4a302c00a260 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:48.874329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ef9cf-1491-45e8-8c01-4a302c00a260 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:48.874602 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ef9cf-1491-45e8-8c01-4a302c00a260 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:48.875222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ef9cf-1491-45e8-8c01-4a302c00a260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:48.877116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ef9cf-1491-45e8-8c01-4a302c00a260 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:48.959999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1250/2520] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:36:48 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:36:48.961272 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e5118-d8f8-4416-bb5d-b491e42cb5df None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:48.972761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e5118-d8f8-4416-bb5d-b491e42cb5df None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:48.984995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:49.029410 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d3fae-8ee0-4233-9b3b-1a2853466319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:49.030070 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d3fae-8ee0-4233-9b3b-1a2853466319 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:49.030420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d3fae-8ee0-4233-9b3b-1a2853466319 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:49.030625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d3fae-8ee0-4233-9b3b-1a2853466319 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:49.031252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d3fae-8ee0-4233-9b3b-1a2853466319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:49.033170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d3fae-8ee0-4233-9b3b-1a2853466319 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:49.121882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1271/2521] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:36:48 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:36:49.125917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ef9cf-1491-45e8-8c01-4a302c00a260 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:49.137452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ef9cf-1491-45e8-8c01-4a302c00a260 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:49.143347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14757a-b8a6-4def-b808-a3dede5b3137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:49.143981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14757a-b8a6-4def-b808-a3dede5b3137 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:49.144255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14757a-b8a6-4def-b808-a3dede5b3137 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:49.144536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14757a-b8a6-4def-b808-a3dede5b3137 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:49.145155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14757a-b8a6-4def-b808-a3dede5b3137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:49.146785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14757a-b8a6-4def-b808-a3dede5b3137 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:49.158504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1251/2522] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:36:49.163145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d3fae-8ee0-4233-9b3b-1a2853466319 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:49.173412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d3fae-8ee0-4233-9b3b-1a2853466319 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:49.178789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e41ed-f0cf-4fb8-998a-571d13a898d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:49.179512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e41ed-f0cf-4fb8-998a-571d13a898d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:49.179858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e41ed-f0cf-4fb8-998a-571d13a898d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:49.180197 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e41ed-f0cf-4fb8-998a-571d13a898d1 admin admin] PATH_INFO: `/v3/projects/441240e16f20478d8b93061c41c77645/users/47af861f8c264e08a8f2ffe4536c9f24/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:49.195282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e41ed-f0cf-4fb8-998a-571d13a898d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441240e16f20478d8b93061c41c77645, user_id=47af861f8c264e08a8f2ffe4536c9f24, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:49.199566 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e41ed-f0cf-4fb8-998a-571d13a898d1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:49.270358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1272/2523] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:36:49 2022] PUT /identity/v3/projects/441240e16f20478d8b93061c41c77645/users/47af861f8c264e08a8f2ffe4536c9f24/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:36:49.274484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14757a-b8a6-4def-b808-a3dede5b3137 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:49.285291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14757a-b8a6-4def-b808-a3dede5b3137 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:49.339470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f44de-22cb-4eef-81f2-7ac786374fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:49.340078 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f44de-22cb-4eef-81f2-7ac786374fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:49.340362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f44de-22cb-4eef-81f2-7ac786374fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:49.340632 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f44de-22cb-4eef-81f2-7ac786374fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:49.341268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f44de-22cb-4eef-81f2-7ac786374fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:49.342940 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f44de-22cb-4eef-81f2-7ac786374fd9 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:49.353924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1252/2524] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:36:49.364350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e41ed-f0cf-4fb8-998a-571d13a898d1 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:49.375301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e41ed-f0cf-4fb8-998a-571d13a898d1 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:49.380262 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed6778-c4bb-41e3-aa0e-870fdfe6a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:49.380926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed6778-c4bb-41e3-aa0e-870fdfe6a69a admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:49.381206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed6778-c4bb-41e3-aa0e-870fdfe6a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:49.381477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed6778-c4bb-41e3-aa0e-870fdfe6a69a admin admin] PATH_INFO: `/v3/projects/441240e16f20478d8b93061c41c77645/users/47af861f8c264e08a8f2ffe4536c9f24/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:49.383495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed6778-c4bb-41e3-aa0e-870fdfe6a69a admin admin] RBAC: Authorizing `identity:create_grant(project_id=441240e16f20478d8b93061c41c77645, user_id=47af861f8c264e08a8f2ffe4536c9f24, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:49.386214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed6778-c4bb-41e3-aa0e-870fdfe6a69a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:49.405864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1273/2525] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:36:49 2022] PUT /identity/v3/projects/441240e16f20478d8b93061c41c77645/users/47af861f8c264e08a8f2ffe4536c9f24/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:49.409898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581aa41-dd08-43a2-a5ea-83bdbddd0609 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:49.413042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581aa41-dd08-43a2-a5ea-83bdbddd0609 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:49.413042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581aa41-dd08-43a2-a5ea-83bdbddd0609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:49.438313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a581aa41-dd08-43a2-a5ea-83bdbddd0609 None None] MFA Rules not processed for user `47af861f8c264e08a8f2ffe4536c9f24`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:49.439523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a581aa41-dd08-43a2-a5ea-83bdbddd0609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:49.513871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1253/2526] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:49.525646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed6778-c4bb-41e3-aa0e-870fdfe6a69a admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:49.527744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-14ed6778-c4bb-41e3-aa0e-870fdfe6a69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:49.586728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed6778-c4bb-41e3-aa0e-870fdfe6a69a admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:49.590392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddd31a-0d45-446a-b385-4fd1fc6eacdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:49.591035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddd31a-0d45-446a-b385-4fd1fc6eacdf service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:49.591307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddd31a-0d45-446a-b385-4fd1fc6eacdf service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:49.591569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddd31a-0d45-446a-b385-4fd1fc6eacdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:49.601670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddd31a-0d45-446a-b385-4fd1fc6eacdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:49.603108 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddd31a-0d45-446a-b385-4fd1fc6eacdf service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:49.612670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1274/2527] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:36:49.715391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581aa41-dd08-43a2-a5ea-83bdbddd0609 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:49.716828 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a581aa41-dd08-43a2-a5ea-83bdbddd0609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:49.729984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581aa41-dd08-43a2-a5ea-83bdbddd0609 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:49.737539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3524a-3bbf-495d-94a8-49e10431c70f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:49.738108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3524a-3bbf-495d-94a8-49e10431c70f service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:49.738637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3524a-3bbf-495d-94a8-49e10431c70f service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:49.738904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3524a-3bbf-495d-94a8-49e10431c70f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:49.751278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:49.801371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3524a-3bbf-495d-94a8-49e10431c70f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:49.801371 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3524a-3bbf-495d-94a8-49e10431c70f service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:49.809902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1254/2528] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:36:52.937811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddd31a-0d45-446a-b385-4fd1fc6eacdf service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:52.999707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddd31a-0d45-446a-b385-4fd1fc6eacdf service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:53.004812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:53.005429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:53.005705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:53.006001 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] PATH_INFO: `/v3/users/1636a19dc30141559c6039addaeccb37` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:53.007103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1636a19dc30141559c6039addaeccb37)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:53.009046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:53.023871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:53.230564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1636a19dc30141559c6039addaeccb37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:53.290741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1636a19dc30141559c6039addaeccb37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:53.298985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1636a19dc30141559c6039addaeccb37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:53.315836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1636a19dc30141559c6039addaeccb37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:53.327565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1636a19dc30141559c6039addaeccb37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:53.327565 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1275/2529] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:36:52 2022] DELETE /identity/v3/users/1636a19dc30141559c6039addaeccb37 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:53.327871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3f341-a05c-4030-b4b1-6e742808fc7f None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:53.328312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3f341-a05c-4030-b4b1-6e742808fc7f None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:53.328312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3f341-a05c-4030-b4b1-6e742808fc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:53.347219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-55c3f341-a05c-4030-b4b1-6e742808fc7f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:53.349989 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-55c3f341-a05c-4030-b4b1-6e742808fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:53.416850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1255/2530] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:36:53.427372 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:53.428819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:53.615461 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a954268-0f94-4ac3-9d59-db4cb91316c9 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:53.619930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b31b5-3f57-4ec0-8066-61bd7a897b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:53.619930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b31b5-3f57-4ec0-8066-61bd7a897b33 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:53.620144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b31b5-3f57-4ec0-8066-61bd7a897b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:53.620558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b31b5-3f57-4ec0-8066-61bd7a897b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:53.629705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b31b5-3f57-4ec0-8066-61bd7a897b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:53.631226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b31b5-3f57-4ec0-8066-61bd7a897b33 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:53.645046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1276/2531] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:36:53.775394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3f341-a05c-4030-b4b1-6e742808fc7f None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:53.779180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-55c3f341-a05c-4030-b4b1-6e742808fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:53.791608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3f341-a05c-4030-b4b1-6e742808fc7f None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:53.795834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576046af-13c4-49a7-9f9b-1bfef7472b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:53.796466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576046af-13c4-49a7-9f9b-1bfef7472b80 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:53.796736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576046af-13c4-49a7-9f9b-1bfef7472b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:53.797000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576046af-13c4-49a7-9f9b-1bfef7472b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:53.798693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-576046af-13c4-49a7-9f9b-1bfef7472b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:53.811112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576046af-13c4-49a7-9f9b-1bfef7472b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:53.813456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576046af-13c4-49a7-9f9b-1bfef7472b80 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:53.823478 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1256/2532] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:36:54.150127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b31b5-3f57-4ec0-8066-61bd7a897b33 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:54.151663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1d2b31b5-3f57-4ec0-8066-61bd7a897b33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:54.168354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b31b5-3f57-4ec0-8066-61bd7a897b33 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:54.173667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:54.174682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:54.175120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:54.175562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] PATH_INFO: `/v3/projects/70edc98b3178483b9352119793b35932` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:54.177074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70edc98b3178483b9352119793b35932)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:54.179360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:54.293060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70edc98b3178483b9352119793b35932', 'request_id': 'req-cb08c518-504f-4473-b5bc-c18317a7b274'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:54.299964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70edc98b3178483b9352119793b35932', 'request_id': 'req-cb08c518-504f-4473-b5bc-c18317a7b274'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:54.446523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70edc98b3178483b9352119793b35932', 'request_id': 'req-cb08c518-504f-4473-b5bc-c18317a7b274'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:54.471495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] The token cache is being invalidate because project 70edc98b3178483b9352119793b35932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:36:54.472161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:54.473062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:54.479687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576046af-13c4-49a7-9f9b-1bfef7472b80 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:54.480916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1277/2533] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:36:54 2022] DELETE /identity/v3/projects/70edc98b3178483b9352119793b35932 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:54.501291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576046af-13c4-49a7-9f9b-1bfef7472b80 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:54.634742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:54.636135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:54.636635 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:54.637231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] PATH_INFO: `/v3/users/47af861f8c264e08a8f2ffe4536c9f24` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:54.638848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47af861f8c264e08a8f2ffe4536c9f24)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:54.640832 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:54.899180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:54.902122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47af861f8c264e08a8f2ffe4536c9f24'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:54.971924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47af861f8c264e08a8f2ffe4536c9f24'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:54.987863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47af861f8c264e08a8f2ffe4536c9f24'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:54.996156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47af861f8c264e08a8f2ffe4536c9f24'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:54.996882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-46699ad9-39b0-429d-b6c3-1adec4312a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47af861f8c264e08a8f2ffe4536c9f24'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:55.004805 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1257/2534] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:36:54 2022] DELETE /identity/v3/users/47af861f8c264e08a8f2ffe4536c9f24 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:55.557441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:55.560819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:55.615015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08c518-504f-4473-b5bc-c18317a7b274 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:55.633614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:55.696715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:55.697358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:55.697636 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:55.697938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] PATH_INFO: `/v3/projects/441240e16f20478d8b93061c41c77645` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:55.699055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=441240e16f20478d8b93061c41c77645)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:55.700945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:55.912445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441240e16f20478d8b93061c41c77645', 'request_id': 'req-bdaf6116-c0bd-49ae-97d2-311a275b25c0'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:55.919435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441240e16f20478d8b93061c41c77645', 'request_id': 'req-bdaf6116-c0bd-49ae-97d2-311a275b25c0'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:56.033394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441240e16f20478d8b93061c41c77645', 'request_id': 'req-bdaf6116-c0bd-49ae-97d2-311a275b25c0'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:56.045708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] The token cache is being invalidate because project 441240e16f20478d8b93061c41c77645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:36:56.046019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:56.047307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-bdaf6116-c0bd-49ae-97d2-311a275b25c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:56.048591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1278/2535] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:36:55 2022] DELETE /identity/v3/projects/441240e16f20478d8b93061c41c77645 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:56.056596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2e638-13d4-45d3-bc7a-7e350e64a4ee None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:56.057109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2e638-13d4-45d3-bc7a-7e350e64a4ee None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:56.057205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2e638-13d4-45d3-bc7a-7e350e64a4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:56.077634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-56f2e638-13d4-45d3-bc7a-7e350e64a4ee None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:56.079182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-56f2e638-13d4-45d3-bc7a-7e350e64a4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:56.149063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1258/2536] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:56.158577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecda000-d2f0-41df-9ecb-3e10b2544a72 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:56.160951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecda000-d2f0-41df-9ecb-3e10b2544a72 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:56.160951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecda000-d2f0-41df-9ecb-3e10b2544a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:56.183467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-aecda000-d2f0-41df-9ecb-3e10b2544a72 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:56.184820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-aecda000-d2f0-41df-9ecb-3e10b2544a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:56.194007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1279/2537] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:56.198230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2e638-13d4-45d3-bc7a-7e350e64a4ee None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:56.208959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2e638-13d4-45d3-bc7a-7e350e64a4ee None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:56.215530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82824c9-dc0e-4c13-b916-d08ee0b848bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:56.216942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82824c9-dc0e-4c13-b916-d08ee0b848bb admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:56.217228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82824c9-dc0e-4c13-b916-d08ee0b848bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:56.217500 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82824c9-dc0e-4c13-b916-d08ee0b848bb admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:56.218218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82824c9-dc0e-4c13-b916-d08ee0b848bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:36:56.218530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82824c9-dc0e-4c13-b916-d08ee0b848bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:56.220124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82824c9-dc0e-4c13-b916-d08ee0b848bb admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:56.243259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1259/2538] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:36:56.243485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecda000-d2f0-41df-9ecb-3e10b2544a72 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:56.252242 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecda000-d2f0-41df-9ecb-3e10b2544a72 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:56.273450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e789d5-02d1-43b8-875f-f2695e25f410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:56.273450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e789d5-02d1-43b8-875f-f2695e25f410 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:56.275011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e789d5-02d1-43b8-875f-f2695e25f410 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:56.275239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e789d5-02d1-43b8-875f-f2695e25f410 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:56.275874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e789d5-02d1-43b8-875f-f2695e25f410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:56.277772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e789d5-02d1-43b8-875f-f2695e25f410 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:56.394982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1280/2539] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:36:56 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:36:56.398965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82824c9-dc0e-4c13-b916-d08ee0b848bb admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:56.410484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82824c9-dc0e-4c13-b916-d08ee0b848bb admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:56.478215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64dd08-7502-40de-a82a-73f4cac21a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:56.478820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64dd08-7502-40de-a82a-73f4cac21a66 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:56.479092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64dd08-7502-40de-a82a-73f4cac21a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:56.479357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64dd08-7502-40de-a82a-73f4cac21a66 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:56.479996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64dd08-7502-40de-a82a-73f4cac21a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:56.481968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64dd08-7502-40de-a82a-73f4cac21a66 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:56.614096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1260/2540] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:36:56 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:36:56.615324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e789d5-02d1-43b8-875f-f2695e25f410 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:56.625652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e789d5-02d1-43b8-875f-f2695e25f410 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:56.630471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bab4f-7b15-439e-878f-e865c948c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:56.631146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bab4f-7b15-439e-878f-e865c948c6af admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:56.631522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bab4f-7b15-439e-878f-e865c948c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:56.633021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bab4f-7b15-439e-878f-e865c948c6af admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:56.633021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534bab4f-7b15-439e-878f-e865c948c6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:56.634732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534bab4f-7b15-439e-878f-e865c948c6af admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:56.648584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1281/2541] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:36:56.650349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64dd08-7502-40de-a82a-73f4cac21a66 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:56.664585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64dd08-7502-40de-a82a-73f4cac21a66 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:56.671203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb451c7-0db4-4496-9231-19a51da28c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:56.671869 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb451c7-0db4-4496-9231-19a51da28c4e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:56.672219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb451c7-0db4-4496-9231-19a51da28c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:56.672893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb451c7-0db4-4496-9231-19a51da28c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:56.673265 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb451c7-0db4-4496-9231-19a51da28c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:56.675876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb451c7-0db4-4496-9231-19a51da28c4e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:56.695514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1261/2542] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:36:56.700051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bab4f-7b15-439e-878f-e865c948c6af admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:56.709331 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bab4f-7b15-439e-878f-e865c948c6af admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:56.714340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eccd2b-5446-4b38-bfef-df884f7cae9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:56.715020 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eccd2b-5446-4b38-bfef-df884f7cae9f admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:56.715546 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eccd2b-5446-4b38-bfef-df884f7cae9f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:56.715826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eccd2b-5446-4b38-bfef-df884f7cae9f admin admin] PATH_INFO: `/v3/projects/7df3aa4fa75f4112aa6bf8d14a94fb3c/users/b2b65ce9b5d741808d5042181575d236/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:56.742890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eccd2b-5446-4b38-bfef-df884f7cae9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df3aa4fa75f4112aa6bf8d14a94fb3c, user_id=b2b65ce9b5d741808d5042181575d236, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:56.746415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eccd2b-5446-4b38-bfef-df884f7cae9f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:56.863586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1282/2543] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:36:56 2022] PUT /identity/v3/projects/7df3aa4fa75f4112aa6bf8d14a94fb3c/users/b2b65ce9b5d741808d5042181575d236/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:36:56.928307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd9361-8230-48e5-9472-604ef3c20460 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:56.928653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd9361-8230-48e5-9472-604ef3c20460 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:56.928934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd9361-8230-48e5-9472-604ef3c20460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:56.957972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-fbfd9361-8230-48e5-9472-604ef3c20460 None None] MFA Rules not processed for user `b2b65ce9b5d741808d5042181575d236`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:56.959189 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-fbfd9361-8230-48e5-9472-604ef3c20460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:56.974068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:57.028384 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1262/2544] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:57.196165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eccd2b-5446-4b38-bfef-df884f7cae9f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:57.198044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-65eccd2b-5446-4b38-bfef-df884f7cae9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:57.253304 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:57.276851 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eccd2b-5446-4b38-bfef-df884f7cae9f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:57.280937 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232b5e5-71bc-4e63-8e64-c3a83982afe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:57.281482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232b5e5-71bc-4e63-8e64-c3a83982afe8 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:57.281862 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232b5e5-71bc-4e63-8e64-c3a83982afe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:57.282310 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232b5e5-71bc-4e63-8e64-c3a83982afe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:57.291897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232b5e5-71bc-4e63-8e64-c3a83982afe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:57.293370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232b5e5-71bc-4e63-8e64-c3a83982afe8 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:57.306218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1283/2545] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:36:57.337349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd9361-8230-48e5-9472-604ef3c20460 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:57.416624 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd9361-8230-48e5-9472-604ef3c20460 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:57.423560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:57.424133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:57.424443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:57.424716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] PATH_INFO: `/v3/users/b2b65ce9b5d741808d5042181575d236` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:57.425892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b65ce9b5d741808d5042181575d236)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:57.429581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:57.663028 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b65ce9b5d741808d5042181575d236'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:57.793422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b65ce9b5d741808d5042181575d236'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:57.808047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b65ce9b5d741808d5042181575d236'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:57.817410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b65ce9b5d741808d5042181575d236'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:57.818169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b65ce9b5d741808d5042181575d236'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:57.824727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1263/2546] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:36:57 2022] DELETE /identity/v3/users/b2b65ce9b5d741808d5042181575d236 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:57.831920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950c35e-1c5c-46cd-b170-3266fa0daa61 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:57.832904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950c35e-1c5c-46cd-b170-3266fa0daa61 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:57.833386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950c35e-1c5c-46cd-b170-3266fa0daa61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:57.861213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3950c35e-1c5c-46cd-b170-3266fa0daa61 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:57.863174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3950c35e-1c5c-46cd-b170-3266fa0daa61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:57.926526 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1284/2547] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:36:57.937897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:57.938464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:57.993292 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:58.011201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093028f-25e1-4ba4-89b3-f3470a719518 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:58.014831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01be17b-2b9c-43cd-a75d-f32fb122b7bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:58.015425 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01be17b-2b9c-43cd-a75d-f32fb122b7bd service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:58.015695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01be17b-2b9c-43cd-a75d-f32fb122b7bd service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:58.015959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01be17b-2b9c-43cd-a75d-f32fb122b7bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:58.025008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01be17b-2b9c-43cd-a75d-f32fb122b7bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:58.026502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01be17b-2b9c-43cd-a75d-f32fb122b7bd service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:58.036641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1264/2548] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:36:58.086578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16d4f-9782-467e-896a-c1161329f66c None None] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:58.086868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16d4f-9782-467e-896a-c1161329f66c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:58.090588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16d4f-9782-467e-896a-c1161329f66c None None] PATH_INFO: `/` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:58.090588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1285/2549] 10.222.0.57 () {54 vars in 921 bytes} [Mon Aug 8 17:36:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 17:36:58.122373 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b379a-d102-4b67-9f42-ec7ab855f566 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:58.122717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b379a-d102-4b67-9f42-ec7ab855f566 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:58.122996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b379a-d102-4b67-9f42-ec7ab855f566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:58.169950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f03b379a-d102-4b67-9f42-ec7ab855f566 None None] MFA Rules not processed for user `5853f9b60cf34380b218a5cda94f587d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:58.171073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f03b379a-d102-4b67-9f42-ec7ab855f566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:58.231005 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1265/2550] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Aug 8 17:36:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:58.242468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16d4f-9782-467e-896a-c1161329f66c None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:58.244241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-23e16d4f-9782-467e-896a-c1161329f66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:58.266454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b379a-d102-4b67-9f42-ec7ab855f566 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:58.275236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16d4f-9782-467e-896a-c1161329f66c None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:58.282002 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a412d-d5fd-4e59-933e-97ef255a23a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:58.282673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f03b379a-d102-4b67-9f42-ec7ab855f566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:58.283321 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a412d-d5fd-4e59-933e-97ef255a23a4 service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:58.283594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a412d-d5fd-4e59-933e-97ef255a23a4 service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:58.284616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a412d-d5fd-4e59-933e-97ef255a23a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:58.294351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a412d-d5fd-4e59-933e-97ef255a23a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:58.297950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a412d-d5fd-4e59-933e-97ef255a23a4 service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:58.309568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b379a-d102-4b67-9f42-ec7ab855f566 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:58.310925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1286/2551] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:36:58.318819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:58.319715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:58.320186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:58.321541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] PATH_INFO: `/v3/users/d6e9bd6db85b4decbdac1908912d158c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:58.324928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e9bd6db85b4decbdac1908912d158c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:58.327559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:58.542171 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e9bd6db85b4decbdac1908912d158c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.563421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a412d-d5fd-4e59-933e-97ef255a23a4 service placement] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:58.563421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-328a412d-d5fd-4e59-933e-97ef255a23a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:58.599901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a412d-d5fd-4e59-933e-97ef255a23a4 service placement] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:58.605001 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:58.605722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:58.606050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:58.606307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] PATH_INFO: `/v3/projects/7df3aa4fa75f4112aa6bf8d14a94fb3c` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:58.607454 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df3aa4fa75f4112aa6bf8d14a94fb3c)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:58.609353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:58.613545 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e9bd6db85b4decbdac1908912d158c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.628412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e9bd6db85b4decbdac1908912d158c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.635791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e9bd6db85b4decbdac1908912d158c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.636793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-443d9c11-09ba-4d34-a69a-ea456cfdf40f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e9bd6db85b4decbdac1908912d158c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.644385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1266/2552] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:36:58 2022] DELETE /identity/v3/users/d6e9bd6db85b4decbdac1908912d158c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:58.775931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df3aa4fa75f4112aa6bf8d14a94fb3c', 'request_id': 'req-8e9cbdd0-1906-481d-97b8-43783b9121e2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.781166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df3aa4fa75f4112aa6bf8d14a94fb3c', 'request_id': 'req-8e9cbdd0-1906-481d-97b8-43783b9121e2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.829696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df3aa4fa75f4112aa6bf8d14a94fb3c', 'request_id': 'req-8e9cbdd0-1906-481d-97b8-43783b9121e2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.840847 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] The token cache is being invalidate because project 7df3aa4fa75f4112aa6bf8d14a94fb3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:36:58.841429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.841905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8e9cbdd0-1906-481d-97b8-43783b9121e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:58.843550 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1287/2553] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:36:58 2022] DELETE /identity/v3/projects/7df3aa4fa75f4112aa6bf8d14a94fb3c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:36:58.849531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83706b6-74ac-4198-8fef-dd1d101251c9 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:58.849839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83706b6-74ac-4198-8fef-dd1d101251c9 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:58.850111 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83706b6-74ac-4198-8fef-dd1d101251c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:58.870059 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d83706b6-74ac-4198-8fef-dd1d101251c9 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:58.871285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d83706b6-74ac-4198-8fef-dd1d101251c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:58.889541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:58.929463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1267/2554] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:58.937366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459c72-39f9-4a54-adc1-ff8d05469ca4 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:58.937646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459c72-39f9-4a54-adc1-ff8d05469ca4 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:58.940329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459c72-39f9-4a54-adc1-ff8d05469ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:58.943865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83706b6-74ac-4198-8fef-dd1d101251c9 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:58.944936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d83706b6-74ac-4198-8fef-dd1d101251c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:58.957432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83706b6-74ac-4198-8fef-dd1d101251c9 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:58.958071 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-69459c72-39f9-4a54-adc1-ff8d05469ca4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:58.960547 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-69459c72-39f9-4a54-adc1-ff8d05469ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:58.963152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:58.963938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:58.964143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:58.964402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] PATH_INFO: `/v3/projects/1a3cc7413c0c4603a7898fd5b0d6a765` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:58.965583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a3cc7413c0c4603a7898fd5b0d6a765)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:58.967549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:58.985009 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1288/2555] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:36:58.987918 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459c72-39f9-4a54-adc1-ff8d05469ca4 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:58.997028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459c72-39f9-4a54-adc1-ff8d05469ca4 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.001715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3301544-e88a-484a-9b78-53f69317b177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.002345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3301544-e88a-484a-9b78-53f69317b177 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.002672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3301544-e88a-484a-9b78-53f69317b177 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.002942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3301544-e88a-484a-9b78-53f69317b177 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.003585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3301544-e88a-484a-9b78-53f69317b177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:36:59.003904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3301544-e88a-484a-9b78-53f69317b177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.005464 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3301544-e88a-484a-9b78-53f69317b177 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.016874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1289/2556] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:36:59.021611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3301544-e88a-484a-9b78-53f69317b177 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.031193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3301544-e88a-484a-9b78-53f69317b177 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.036367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6b680-3510-4538-a4b4-b8362483c754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.036940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6b680-3510-4538-a4b4-b8362483c754 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.037214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6b680-3510-4538-a4b4-b8362483c754 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.037484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6b680-3510-4538-a4b4-b8362483c754 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.038120 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6b680-3510-4538-a4b4-b8362483c754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.039952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6b680-3510-4538-a4b4-b8362483c754 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.090980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a3cc7413c0c4603a7898fd5b0d6a765', 'request_id': 'req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:59.097600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a3cc7413c0c4603a7898fd5b0d6a765', 'request_id': 'req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:59.156981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1290/2557] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:36:59 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:36:59.161710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6b680-3510-4538-a4b4-b8362483c754 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.172629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a3cc7413c0c4603a7898fd5b0d6a765', 'request_id': 'req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:59.178645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:59.183963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] The token cache is being invalidate because project 1a3cc7413c0c4603a7898fd5b0d6a765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:36:59.184333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:59.184983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b2c45ba8-8c4e-472b-ab2a-fc1bd3a6ffb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:36:59.186548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1268/2558] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:36:58 2022] DELETE /identity/v3/projects/1a3cc7413c0c4603a7898fd5b0d6a765 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:36:59.192305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f6e3a-6771-4d0b-8a6d-749f631c8598 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.192528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f6e3a-6771-4d0b-8a6d-749f631c8598 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.192798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f6e3a-6771-4d0b-8a6d-749f631c8598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.212212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-d99f6e3a-6771-4d0b-8a6d-749f631c8598 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:59.213410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d99f6e3a-6771-4d0b-8a6d-749f631c8598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:59.227668 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6b680-3510-4538-a4b4-b8362483c754 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.233523 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9655f-745c-4173-88a5-b02ce00470a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.234799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9655f-745c-4173-88a5-b02ce00470a5 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.234799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9655f-745c-4173-88a5-b02ce00470a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.235237 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9655f-745c-4173-88a5-b02ce00470a5 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.235890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9655f-745c-4173-88a5-b02ce00470a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.237983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9655f-745c-4173-88a5-b02ce00470a5 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.273317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1269/2559] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:36:59.337185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94714d64-fbee-4508-980d-c5c4e0ce28e7 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.337466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94714d64-fbee-4508-980d-c5c4e0ce28e7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.337712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94714d64-fbee-4508-980d-c5c4e0ce28e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.342278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1291/2560] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:36:59 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:36:59.343270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9655f-745c-4173-88a5-b02ce00470a5 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.344878 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-45e9655f-745c-4173-88a5-b02ce00470a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:59.358331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-94714d64-fbee-4508-980d-c5c4e0ce28e7 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:59.365856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9655f-745c-4173-88a5-b02ce00470a5 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.365982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-94714d64-fbee-4508-980d-c5c4e0ce28e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:59.366087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92648d6-61dd-43a3-b33e-67ba3646b269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.366087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92648d6-61dd-43a3-b33e-67ba3646b269 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.366087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92648d6-61dd-43a3-b33e-67ba3646b269 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.366087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92648d6-61dd-43a3-b33e-67ba3646b269 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.366530 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92648d6-61dd-43a3-b33e-67ba3646b269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.367013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92648d6-61dd-43a3-b33e-67ba3646b269 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.368068 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1270/2561] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:36:59.372220 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94714d64-fbee-4508-980d-c5c4e0ce28e7 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.381830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1292/2562] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:36:59.382001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94714d64-fbee-4508-980d-c5c4e0ce28e7 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.383767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92648d6-61dd-43a3-b33e-67ba3646b269 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.386594 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677e993-1ce4-4de2-b5a2-bfdd4f80c429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.388546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677e993-1ce4-4de2-b5a2-bfdd4f80c429 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.388546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677e993-1ce4-4de2-b5a2-bfdd4f80c429 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.388546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677e993-1ce4-4de2-b5a2-bfdd4f80c429 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.388546 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677e993-1ce4-4de2-b5a2-bfdd4f80c429 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:36:59.388878 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677e993-1ce4-4de2-b5a2-bfdd4f80c429 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.390223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677e993-1ce4-4de2-b5a2-bfdd4f80c429 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.394574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92648d6-61dd-43a3-b33e-67ba3646b269 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.399252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d9fa1-1243-4b11-882c-4c7a9349e97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.399832 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d9fa1-1243-4b11-882c-4c7a9349e97e admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.400206 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d9fa1-1243-4b11-882c-4c7a9349e97e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.400474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d9fa1-1243-4b11-882c-4c7a9349e97e admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.401094 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d9fa1-1243-4b11-882c-4c7a9349e97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.402703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1271/2563] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:36:59.406382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d9fa1-1243-4b11-882c-4c7a9349e97e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.407178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677e993-1ce4-4de2-b5a2-bfdd4f80c429 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.414174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1293/2564] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:36:59.416984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677e993-1ce4-4de2-b5a2-bfdd4f80c429 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.418016 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d9fa1-1243-4b11-882c-4c7a9349e97e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.421758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49147b7-bef0-4492-97f8-3d7765477052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.422376 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49147b7-bef0-4492-97f8-3d7765477052 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.422650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49147b7-bef0-4492-97f8-3d7765477052 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.423033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49147b7-bef0-4492-97f8-3d7765477052 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.423648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49147b7-bef0-4492-97f8-3d7765477052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.425618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49147b7-bef0-4492-97f8-3d7765477052 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.427527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d9fa1-1243-4b11-882c-4c7a9349e97e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.432590 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787daba1-a287-4290-80d5-554272d902d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.433211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787daba1-a287-4290-80d5-554272d902d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.433484 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787daba1-a287-4290-80d5-554272d902d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.433751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787daba1-a287-4290-80d5-554272d902d5 admin admin] PATH_INFO: `/v3/projects/7656efe595ac421b95e3df807f7ac434/users/55a8c2e9fedb4b41a17a48bdac90c912/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.449873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787daba1-a287-4290-80d5-554272d902d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7656efe595ac421b95e3df807f7ac434, user_id=55a8c2e9fedb4b41a17a48bdac90c912, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.452451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787daba1-a287-4290-80d5-554272d902d5 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.516640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1272/2565] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:36:59 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:36:59.521269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49147b7-bef0-4492-97f8-3d7765477052 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.529576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:59.538402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1294/2566] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:36:59 2022] PUT /identity/v3/projects/7656efe595ac421b95e3df807f7ac434/users/55a8c2e9fedb4b41a17a48bdac90c912/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:36:59.549408 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76f7d-b2a2-4fe8-894e-cb1be5cca61e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.549692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76f7d-b2a2-4fe8-894e-cb1be5cca61e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.549987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c76f7d-b2a2-4fe8-894e-cb1be5cca61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.579146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-17c76f7d-b2a2-4fe8-894e-cb1be5cca61e None None] MFA Rules not processed for user `55a8c2e9fedb4b41a17a48bdac90c912`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:36:59.580342 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-17c76f7d-b2a2-4fe8-894e-cb1be5cca61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:59.597950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49147b7-bef0-4492-97f8-3d7765477052 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.647227 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1295/2567] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:36:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:36:59.651676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76f7d-b2a2-4fe8-894e-cb1be5cca61e None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.660060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607a6b1-b08e-46d0-bb41-7dfa33e5d94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.660662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607a6b1-b08e-46d0-bb41-7dfa33e5d94a admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.660935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607a6b1-b08e-46d0-bb41-7dfa33e5d94a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.661201 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607a6b1-b08e-46d0-bb41-7dfa33e5d94a admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.662046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607a6b1-b08e-46d0-bb41-7dfa33e5d94a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.664304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607a6b1-b08e-46d0-bb41-7dfa33e5d94a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.703907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:59.717987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c76f7d-b2a2-4fe8-894e-cb1be5cca61e None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.722599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61402278-21ce-42b4-9a88-c60813d4de75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.723222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61402278-21ce-42b4-9a88-c60813d4de75 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.723786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61402278-21ce-42b4-9a88-c60813d4de75 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.724052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61402278-21ce-42b4-9a88-c60813d4de75 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.724666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61402278-21ce-42b4-9a88-c60813d4de75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.726601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61402278-21ce-42b4-9a88-c60813d4de75 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.743076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1273/2568] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:36:59 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:36:59.743076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607a6b1-b08e-46d0-bb41-7dfa33e5d94a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.744039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e607a6b1-b08e-46d0-bb41-7dfa33e5d94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:59.800817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607a6b1-b08e-46d0-bb41-7dfa33e5d94a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.804555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9d8e4-c272-42e1-afaf-0d5ae2ddb7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.805288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9d8e4-c272-42e1-afaf-0d5ae2ddb7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.805566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9d8e4-c272-42e1-afaf-0d5ae2ddb7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.805863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9d8e4-c272-42e1-afaf-0d5ae2ddb7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.807618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-abc9d8e4-c272-42e1-afaf-0d5ae2ddb7e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:36:59.820256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9d8e4-c272-42e1-afaf-0d5ae2ddb7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.825167 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9d8e4-c272-42e1-afaf-0d5ae2ddb7e4 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.829771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1296/2569] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:36:59 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:36:59.830650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61402278-21ce-42b4-9a88-c60813d4de75 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.831475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1274/2570] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:36:59.833579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9d8e4-c272-42e1-afaf-0d5ae2ddb7e4 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.844796 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9d8e4-c272-42e1-afaf-0d5ae2ddb7e4 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.905492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61402278-21ce-42b4-9a88-c60813d4de75 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.907394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:36:59.911335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959d511-23d6-45c6-955e-bc7278d1f228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.912171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959d511-23d6-45c6-955e-bc7278d1f228 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.916659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959d511-23d6-45c6-955e-bc7278d1f228 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.926326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959d511-23d6-45c6-955e-bc7278d1f228 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.926326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959d511-23d6-45c6-955e-bc7278d1f228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.926326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959d511-23d6-45c6-955e-bc7278d1f228 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.932133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1297/2571] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:36:59.935238 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7bec2-610e-4c64-9e10-c3d34aa48ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.935865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7bec2-610e-4c64-9e10-c3d34aa48ede admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.936278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7bec2-610e-4c64-9e10-c3d34aa48ede admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.936447 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959d511-23d6-45c6-955e-bc7278d1f228 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.936573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7bec2-610e-4c64-9e10-c3d34aa48ede admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.937166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7bec2-610e-4c64-9e10-c3d34aa48ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.939729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7bec2-610e-4c64-9e10-c3d34aa48ede admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.956871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959d511-23d6-45c6-955e-bc7278d1f228 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:36:59.961167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650e180-9098-4072-82fd-0c9f953f601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:36:59.961167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650e180-9098-4072-82fd-0c9f953f601c admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:36:59.961167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650e180-9098-4072-82fd-0c9f953f601c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:36:59.961167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650e180-9098-4072-82fd-0c9f953f601c admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:36:59.962330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650e180-9098-4072-82fd-0c9f953f601c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:36:59.962435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650e180-9098-4072-82fd-0c9f953f601c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:36:59.974597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1298/2572] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:36:59.975826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650e180-9098-4072-82fd-0c9f953f601c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:36:59.977705 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6650e180-9098-4072-82fd-0c9f953f601c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:00.026822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1275/2573] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:36:59 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:37:00.027775 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7bec2-610e-4c64-9e10-c3d34aa48ede admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:00.039930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7bec2-610e-4c64-9e10-c3d34aa48ede admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:00.040893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650e180-9098-4072-82fd-0c9f953f601c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:00.044703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e0caa-05d4-470b-a42d-fcedffd90d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:00.045321 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e0caa-05d4-470b-a42d-fcedffd90d41 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.045986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e0caa-05d4-470b-a42d-fcedffd90d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.046101 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792f604-fbd0-4e95-81a1-17a3795c32c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:00.046218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e0caa-05d4-470b-a42d-fcedffd90d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.046430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792f604-fbd0-4e95-81a1-17a3795c32c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.046691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792f604-fbd0-4e95-81a1-17a3795c32c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.046959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792f604-fbd0-4e95-81a1-17a3795c32c8 admin admin] PATH_INFO: `/v3/projects/e449345a18254a1ab181cbbcfafa33c8/users/054f2a730eaa4a32b11f4a731c687094/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.047814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ce1e0caa-05d4-470b-a42d-fcedffd90d41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:00.060349 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e0caa-05d4-470b-a42d-fcedffd90d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:00.061473 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792f604-fbd0-4e95-81a1-17a3795c32c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e449345a18254a1ab181cbbcfafa33c8, user_id=054f2a730eaa4a32b11f4a731c687094, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:00.061753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e0caa-05d4-470b-a42d-fcedffd90d41 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:00.064233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792f604-fbd0-4e95-81a1-17a3795c32c8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:00.071504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1299/2574] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:00.072288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e0caa-05d4-470b-a42d-fcedffd90d41 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:00.081681 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e0caa-05d4-470b-a42d-fcedffd90d41 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:00.086208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33687851-d560-46d5-ab7e-1c54fd92fe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:00.086786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33687851-d560-46d5-ab7e-1c54fd92fe7b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.087060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33687851-d560-46d5-ab7e-1c54fd92fe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.087323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33687851-d560-46d5-ab7e-1c54fd92fe7b admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.087968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33687851-d560-46d5-ab7e-1c54fd92fe7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:00.089528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33687851-d560-46d5-ab7e-1c54fd92fe7b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:00.101805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1300/2575] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:00.105956 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33687851-d560-46d5-ab7e-1c54fd92fe7b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:00.116049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33687851-d560-46d5-ab7e-1c54fd92fe7b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:00.121147 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac7492-754c-4972-9164-46590be99ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:00.121881 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac7492-754c-4972-9164-46590be99ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.122692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac7492-754c-4972-9164-46590be99ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.122692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac7492-754c-4972-9164-46590be99ee6 admin admin] PATH_INFO: `/v3/projects/be0d6d0884f84fb5b21e7f9e0178046e/users/78cf9fc6e0d44fc3950c9ad9c5346cfa/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.146243 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1276/2576] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:00 2022] PUT /identity/v3/projects/e449345a18254a1ab181cbbcfafa33c8/users/054f2a730eaa4a32b11f4a731c687094/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:00.146675 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac7492-754c-4972-9164-46590be99ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0d6d0884f84fb5b21e7f9e0178046e, user_id=78cf9fc6e0d44fc3950c9ad9c5346cfa, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:00.151230 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac7492-754c-4972-9164-46590be99ee6 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:00.152748 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ff9e-4dc5-429c-b73c-328c3d79bbd6 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.153608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ff9e-4dc5-429c-b73c-328c3d79bbd6 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.153608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ff9e-4dc5-429c-b73c-328c3d79bbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.183514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c5c6ff9e-4dc5-429c-b73c-328c3d79bbd6 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:00.184779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c5c6ff9e-4dc5-429c-b73c-328c3d79bbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:00.250428 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1301/2577] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:00 2022] PUT /identity/v3/projects/be0d6d0884f84fb5b21e7f9e0178046e/users/78cf9fc6e0d44fc3950c9ad9c5346cfa/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:37:00.256250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac7492-754c-4972-9164-46590be99ee6 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:00.267803 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1277/2578] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:00.278548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ff9e-4dc5-429c-b73c-328c3d79bbd6 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:00.300624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:00.353384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac7492-754c-4972-9164-46590be99ee6 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:00.358948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ff9e-4dc5-429c-b73c-328c3d79bbd6 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:00.360534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf3713-02d6-4b46-a555-31817f8b9200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:00.361368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf3713-02d6-4b46-a555-31817f8b9200 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.361714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf3713-02d6-4b46-a555-31817f8b9200 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.362113 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf3713-02d6-4b46-a555-31817f8b9200 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.363000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf3713-02d6-4b46-a555-31817f8b9200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:00.366612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf3713-02d6-4b46-a555-31817f8b9200 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:00.390008 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1302/2579] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:37:00.392075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf3713-02d6-4b46-a555-31817f8b9200 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:00.407031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf3713-02d6-4b46-a555-31817f8b9200 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:00.412138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f3709-ef17-4c2a-a41a-319d8b4a3c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:00.420509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f3709-ef17-4c2a-a41a-319d8b4a3c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.420509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f3709-ef17-4c2a-a41a-319d8b4a3c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.420509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f3709-ef17-4c2a-a41a-319d8b4a3c9e admin admin] PATH_INFO: `/v3/projects/be0d6d0884f84fb5b21e7f9e0178046e/users/78cf9fc6e0d44fc3950c9ad9c5346cfa/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.420509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f3709-ef17-4c2a-a41a-319d8b4a3c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0d6d0884f84fb5b21e7f9e0178046e, user_id=78cf9fc6e0d44fc3950c9ad9c5346cfa, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:00.420509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f3709-ef17-4c2a-a41a-319d8b4a3c9e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:00.421087 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649ef42-739b-49bf-9393-9d3f19183d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:00.421376 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649ef42-739b-49bf-9393-9d3f19183d2d service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.422026 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649ef42-739b-49bf-9393-9d3f19183d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.422026 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649ef42-739b-49bf-9393-9d3f19183d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.438895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1303/2580] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:00 2022] PUT /identity/v3/projects/be0d6d0884f84fb5b21e7f9e0178046e/users/78cf9fc6e0d44fc3950c9ad9c5346cfa/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:37:00.444117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649ef42-739b-49bf-9393-9d3f19183d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:00.445639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649ef42-739b-49bf-9393-9d3f19183d2d service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:00.451456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28955738-42dc-4434-9e91-3dac1aa1148e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.454624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28955738-42dc-4434-9e91-3dac1aa1148e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.454624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28955738-42dc-4434-9e91-3dac1aa1148e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.457950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1278/2581] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:00.492620 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-28955738-42dc-4434-9e91-3dac1aa1148e None None] MFA Rules not processed for user `78cf9fc6e0d44fc3950c9ad9c5346cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:00.493912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-28955738-42dc-4434-9e91-3dac1aa1148e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:00.576870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1304/2582] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:00.588855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649ef42-739b-49bf-9393-9d3f19183d2d service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:00.591488 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3649ef42-739b-49bf-9393-9d3f19183d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:00.598867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28955738-42dc-4434-9e91-3dac1aa1148e None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:00.600266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-28955738-42dc-4434-9e91-3dac1aa1148e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:00.613228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:00.665539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649ef42-739b-49bf-9393-9d3f19183d2d service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:00.688290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28955738-42dc-4434-9e91-3dac1aa1148e None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:00.694414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:00.695256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.697903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.697903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] PATH_INFO: `/v3/users/e941c29e3df944a08f18f5e029a28977` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.697903 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e941c29e3df944a08f18f5e029a28977)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:00.699393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:00.734819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98decc6c-aeff-427c-b570-fc55ca955387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:00.735478 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98decc6c-aeff-427c-b570-fc55ca955387 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:00.735753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98decc6c-aeff-427c-b570-fc55ca955387 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:00.736018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98decc6c-aeff-427c-b570-fc55ca955387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:00.798568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98decc6c-aeff-427c-b570-fc55ca955387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:00.800572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98decc6c-aeff-427c-b570-fc55ca955387 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:00.809908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1279/2583] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:00.961898 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:00.974628 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e941c29e3df944a08f18f5e029a28977'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.044714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e941c29e3df944a08f18f5e029a28977'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.052951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e941c29e3df944a08f18f5e029a28977'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.065256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e941c29e3df944a08f18f5e029a28977'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.066121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-02f73d7f-829b-4440-a916-a8d75c31af48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e941c29e3df944a08f18f5e029a28977'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.072689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1305/2584] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:37:00 2022] DELETE /identity/v3/users/e941c29e3df944a08f18f5e029a28977 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:37:01.512265 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98decc6c-aeff-427c-b570-fc55ca955387 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:01.527771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98decc6c-aeff-427c-b570-fc55ca955387 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:01.586978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:01.587610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:01.587889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:01.588399 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] PATH_INFO: `/v3/projects/63b568870ecc4e4797919608bd9273ad` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:01.589465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63b568870ecc4e4797919608bd9273ad)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:01.591525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:01.622585 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:01.729552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63b568870ecc4e4797919608bd9273ad', 'request_id': 'req-e33b460c-1014-4e13-af93-8e9519a67b45'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.738535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63b568870ecc4e4797919608bd9273ad', 'request_id': 'req-e33b460c-1014-4e13-af93-8e9519a67b45'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.825291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63b568870ecc4e4797919608bd9273ad', 'request_id': 'req-e33b460c-1014-4e13-af93-8e9519a67b45'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.843339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] The token cache is being invalidate because project 63b568870ecc4e4797919608bd9273ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:37:01.843609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.844872 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e33b460c-1014-4e13-af93-8e9519a67b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:01.852562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1280/2585] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:37:01 2022] DELETE /identity/v3/projects/63b568870ecc4e4797919608bd9273ad => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:01.852771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62789fe-3cde-4102-9713-dd1f873836bc None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:01.852977 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62789fe-3cde-4102-9713-dd1f873836bc None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:01.853255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62789fe-3cde-4102-9713-dd1f873836bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:01.881046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-b62789fe-3cde-4102-9713-dd1f873836bc None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:01.882329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b62789fe-3cde-4102-9713-dd1f873836bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:01.947783 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1306/2586] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:01.962305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee72b9-b84c-4e69-b361-4238f3369f4d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:01.962600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee72b9-b84c-4e69-b361-4238f3369f4d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:01.962906 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee72b9-b84c-4e69-b361-4238f3369f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:01.982586 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ccee72b9-b84c-4e69-b361-4238f3369f4d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:01.984031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ccee72b9-b84c-4e69-b361-4238f3369f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:01.992151 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1281/2587] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:01.996440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62789fe-3cde-4102-9713-dd1f873836bc None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:02.006931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62789fe-3cde-4102-9713-dd1f873836bc None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:02.011520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16b460-e63d-4b39-9284-feb430c1db0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:02.012124 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b460-e63d-4b39-9284-feb430c1db0d admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:02.012401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b460-e63d-4b39-9284-feb430c1db0d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:02.012672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b460-e63d-4b39-9284-feb430c1db0d admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:02.013348 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16b460-e63d-4b39-9284-feb430c1db0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:37:02.013647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16b460-e63d-4b39-9284-feb430c1db0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:02.015292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16b460-e63d-4b39-9284-feb430c1db0d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:02.031291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1307/2588] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:37:02.032666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee72b9-b84c-4e69-b361-4238f3369f4d None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:02.058460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee72b9-b84c-4e69-b361-4238f3369f4d None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:02.078004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e8a0a-5f8e-4d7d-b1a8-28662f069da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:02.078004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e8a0a-5f8e-4d7d-b1a8-28662f069da9 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:02.078004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e8a0a-5f8e-4d7d-b1a8-28662f069da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:02.078004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e8a0a-5f8e-4d7d-b1a8-28662f069da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:02.078480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e8a0a-5f8e-4d7d-b1a8-28662f069da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:02.078480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e8a0a-5f8e-4d7d-b1a8-28662f069da9 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:02.187513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1282/2589] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:02 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:02.192458 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16b460-e63d-4b39-9284-feb430c1db0d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:02.205870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16b460-e63d-4b39-9284-feb430c1db0d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:02.311820 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b2276-2a3f-4892-b58c-69bf3406b650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:02.312631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b2276-2a3f-4892-b58c-69bf3406b650 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:02.312927 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b2276-2a3f-4892-b58c-69bf3406b650 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:02.313202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b2276-2a3f-4892-b58c-69bf3406b650 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:02.314594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b2276-2a3f-4892-b58c-69bf3406b650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:02.318916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b2276-2a3f-4892-b58c-69bf3406b650 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:02.407790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1308/2590] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:02 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:02.412393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e8a0a-5f8e-4d7d-b1a8-28662f069da9 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:02.423482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e8a0a-5f8e-4d7d-b1a8-28662f069da9 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:02.429241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674c94-46de-480f-b843-cbdbc6161661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:02.429904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674c94-46de-480f-b843-cbdbc6161661 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:02.430193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674c94-46de-480f-b843-cbdbc6161661 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:02.430453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674c94-46de-480f-b843-cbdbc6161661 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:02.431115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e674c94-46de-480f-b843-cbdbc6161661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:02.432773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e674c94-46de-480f-b843-cbdbc6161661 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:02.449929 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b2276-2a3f-4892-b58c-69bf3406b650 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:02.456224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1283/2591] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:02.464539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b2276-2a3f-4892-b58c-69bf3406b650 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:02.470596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465eab3a-b2c5-458b-8146-0b637f9c90cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:02.471880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465eab3a-b2c5-458b-8146-0b637f9c90cb admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:02.472402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465eab3a-b2c5-458b-8146-0b637f9c90cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:02.472781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465eab3a-b2c5-458b-8146-0b637f9c90cb admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:02.473874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465eab3a-b2c5-458b-8146-0b637f9c90cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:02.475701 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465eab3a-b2c5-458b-8146-0b637f9c90cb admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:02.494634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674c94-46de-480f-b843-cbdbc6161661 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:02.497873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1309/2592] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:02.510166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674c94-46de-480f-b843-cbdbc6161661 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:02.517902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67796e5-0498-41ab-9aa0-c474f497d17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:02.518733 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67796e5-0498-41ab-9aa0-c474f497d17b admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:02.519014 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67796e5-0498-41ab-9aa0-c474f497d17b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:02.519288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67796e5-0498-41ab-9aa0-c474f497d17b admin admin] PATH_INFO: `/v3/projects/0f857768418e43d8bdbab85be767b896/users/57662358169d4830b2ebb7d594efb701/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:02.537662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67796e5-0498-41ab-9aa0-c474f497d17b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f857768418e43d8bdbab85be767b896, user_id=57662358169d4830b2ebb7d594efb701, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:02.540934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67796e5-0498-41ab-9aa0-c474f497d17b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:02.639222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1284/2593] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:02 2022] PUT /identity/v3/projects/0f857768418e43d8bdbab85be767b896/users/57662358169d4830b2ebb7d594efb701/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:37:02.647062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851fe9f-6533-4de9-998f-97aa5e7d0a11 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:02.647527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851fe9f-6533-4de9-998f-97aa5e7d0a11 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:02.648019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851fe9f-6533-4de9-998f-97aa5e7d0a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:02.689066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f851fe9f-6533-4de9-998f-97aa5e7d0a11 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:02.693721 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f851fe9f-6533-4de9-998f-97aa5e7d0a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:02.735650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:02.797213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1310/2594] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:02.809445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67796e5-0498-41ab-9aa0-c474f497d17b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:02.809445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b67796e5-0498-41ab-9aa0-c474f497d17b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:02.889700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67796e5-0498-41ab-9aa0-c474f497d17b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:02.894191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59fea3-7cb0-496c-af75-a8e59fd7b5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:02.895027 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59fea3-7cb0-496c-af75-a8e59fd7b5ea service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:02.895360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59fea3-7cb0-496c-af75-a8e59fd7b5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:02.896109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59fea3-7cb0-496c-af75-a8e59fd7b5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:02.906117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59fea3-7cb0-496c-af75-a8e59fd7b5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:02.907662 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59fea3-7cb0-496c-af75-a8e59fd7b5ea service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:02.920282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1285/2595] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:37:04.078693 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851fe9f-6533-4de9-998f-97aa5e7d0a11 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:04.080337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f851fe9f-6533-4de9-998f-97aa5e7d0a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:04.100104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851fe9f-6533-4de9-998f-97aa5e7d0a11 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:04.105880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:04.106637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:04.107005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:04.107370 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] PATH_INFO: `/v3/users/55a8c2e9fedb4b41a17a48bdac90c912` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:04.109155 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a8c2e9fedb4b41a17a48bdac90c912)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:04.111196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:04.375911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a8c2e9fedb4b41a17a48bdac90c912'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:04.438957 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a8c2e9fedb4b41a17a48bdac90c912'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:04.449760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a8c2e9fedb4b41a17a48bdac90c912'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:04.482858 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a8c2e9fedb4b41a17a48bdac90c912'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:04.484214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a8c2e9fedb4b41a17a48bdac90c912'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:04.494885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1311/2596] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:37:04 2022] DELETE /identity/v3/users/55a8c2e9fedb4b41a17a48bdac90c912 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:04.496880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18074555-49d1-4f66-a394-c79e4dd97506 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:04.497362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18074555-49d1-4f66-a394-c79e4dd97506 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:04.497465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18074555-49d1-4f66-a394-c79e4dd97506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:04.536472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-18074555-49d1-4f66-a394-c79e4dd97506 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:04.537761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-18074555-49d1-4f66-a394-c79e4dd97506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:04.624926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:04.641286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1286/2597] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:04.652521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:04.654472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:04.764158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c85c3b-4495-434c-95ee-35d47a30f48a admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:04.778752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb3028-23c0-46c9-95a1-f6c72f16514b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:04.779355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb3028-23c0-46c9-95a1-f6c72f16514b service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:04.779638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb3028-23c0-46c9-95a1-f6c72f16514b service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:04.779913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb3028-23c0-46c9-95a1-f6c72f16514b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:04.802888 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb3028-23c0-46c9-95a1-f6c72f16514b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:04.804682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb3028-23c0-46c9-95a1-f6c72f16514b service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:04.824409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1312/2598] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:05.916486 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18074555-49d1-4f66-a394-c79e4dd97506 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:05.917719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-18074555-49d1-4f66-a394-c79e4dd97506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:05.951602 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18074555-49d1-4f66-a394-c79e4dd97506 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:05.970476 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:05.971466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:05.971991 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:05.972269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] PATH_INFO: `/v3/projects/7656efe595ac421b95e3df807f7ac434` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:05.973754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7656efe595ac421b95e3df807f7ac434)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:05.976170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:06.154398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7656efe595ac421b95e3df807f7ac434', 'request_id': 'req-47d370b1-228f-43f3-830d-28d6e2c3c590'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.169588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7656efe595ac421b95e3df807f7ac434', 'request_id': 'req-47d370b1-228f-43f3-830d-28d6e2c3c590'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.316034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7656efe595ac421b95e3df807f7ac434', 'request_id': 'req-47d370b1-228f-43f3-830d-28d6e2c3c590'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.360880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] The token cache is being invalidate because project 7656efe595ac421b95e3df807f7ac434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:37:06.361188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.363280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.369736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1287/2599] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:37:05 2022] DELETE /identity/v3/projects/7656efe595ac421b95e3df807f7ac434 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:37:06.370684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb3028-23c0-46c9-95a1-f6c72f16514b service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:06.387290 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb3028-23c0-46c9-95a1-f6c72f16514b service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:06.464065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:06.496130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:06.496771 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:06.497175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:06.497501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] PATH_INFO: `/v3/users/78cf9fc6e0d44fc3950c9ad9c5346cfa` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:06.498636 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=78cf9fc6e0d44fc3950c9ad9c5346cfa)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:06.500643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:06.681941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:06.683410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:06.702889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d370b1-228f-43f3-830d-28d6e2c3c590 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:06.716099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:06.716956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:06.717244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:06.717516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] PATH_INFO: `/v3/users/5a7ed5f2559a449fbe0b30bfeabd90df` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:06.729463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7ed5f2559a449fbe0b30bfeabd90df)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:06.734905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:06.759885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78cf9fc6e0d44fc3950c9ad9c5346cfa'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.875855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78cf9fc6e0d44fc3950c9ad9c5346cfa'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.886339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78cf9fc6e0d44fc3950c9ad9c5346cfa'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.913293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78cf9fc6e0d44fc3950c9ad9c5346cfa'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.914129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78cf9fc6e0d44fc3950c9ad9c5346cfa'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:06.926686 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1313/2600] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:37:06 2022] DELETE /identity/v3/users/78cf9fc6e0d44fc3950c9ad9c5346cfa => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:07.041538 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7ed5f2559a449fbe0b30bfeabd90df'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:07.178939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7ed5f2559a449fbe0b30bfeabd90df'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:07.199135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7ed5f2559a449fbe0b30bfeabd90df'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:07.208316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7ed5f2559a449fbe0b30bfeabd90df'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:07.209114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7ed5f2559a449fbe0b30bfeabd90df'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:07.215501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1288/2601] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:37:06 2022] DELETE /identity/v3/users/5a7ed5f2559a449fbe0b30bfeabd90df => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:07.763455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:07.765516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:07.945164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:07.946641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:07.965728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb6f4e-2147-426f-88d6-174f87c780bc admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:08.081894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:08.130686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435bb10-66c8-4b9f-aeaf-b062e8f32dfb admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:08.137205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:08.137886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:08.138169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:08.138514 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] PATH_INFO: `/v3/projects/be0d6d0884f84fb5b21e7f9e0178046e` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:08.141910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0d6d0884f84fb5b21e7f9e0178046e)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:08.144374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:08.146486 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:08.147412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:08.147936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:08.152081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] PATH_INFO: `/v3/projects/880cab17504d403d89b8cfff76632c6b` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:08.153522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:08.154661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880cab17504d403d89b8cfff76632c6b)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:08.159705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:08.354491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880cab17504d403d89b8cfff76632c6b', 'request_id': 'req-abc4bac1-06c2-471d-b638-8579c1fa8612'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.359914 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0d6d0884f84fb5b21e7f9e0178046e', 'request_id': 'req-4a7b0918-1671-4da3-a966-14970a3b40ab'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.364673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880cab17504d403d89b8cfff76632c6b', 'request_id': 'req-abc4bac1-06c2-471d-b638-8579c1fa8612'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.369959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0d6d0884f84fb5b21e7f9e0178046e', 'request_id': 'req-4a7b0918-1671-4da3-a966-14970a3b40ab'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.490403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880cab17504d403d89b8cfff76632c6b', 'request_id': 'req-abc4bac1-06c2-471d-b638-8579c1fa8612'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.506042 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] The token cache is being invalidate because project 880cab17504d403d89b8cfff76632c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:37:08.506664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.507893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-abc4bac1-06c2-471d-b638-8579c1fa8612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.510820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1289/2602] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:37:07 2022] DELETE /identity/v3/projects/880cab17504d403d89b8cfff76632c6b => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:08.515657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58e403-e00a-4707-b2e3-d3c1a78da8b8 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:08.519590 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58e403-e00a-4707-b2e3-d3c1a78da8b8 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:08.520089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58e403-e00a-4707-b2e3-d3c1a78da8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:08.554056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dd58e403-e00a-4707-b2e3-d3c1a78da8b8 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:08.555834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dd58e403-e00a-4707-b2e3-d3c1a78da8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:08.568035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0d6d0884f84fb5b21e7f9e0178046e', 'request_id': 'req-4a7b0918-1671-4da3-a966-14970a3b40ab'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.586459 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] The token cache is being invalidate because project be0d6d0884f84fb5b21e7f9e0178046e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:37:08.587035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.588488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4a7b0918-1671-4da3-a966-14970a3b40ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:08.593000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1314/2603] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:37:07 2022] DELETE /identity/v3/projects/be0d6d0884f84fb5b21e7f9e0178046e => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:08.597980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefec8e-547a-4d53-b428-9d66adfa0f1e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:08.598187 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefec8e-547a-4d53-b428-9d66adfa0f1e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:08.598317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefec8e-547a-4d53-b428-9d66adfa0f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:08.635980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-afefec8e-547a-4d53-b428-9d66adfa0f1e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:08.638656 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-afefec8e-547a-4d53-b428-9d66adfa0f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:08.663347 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1290/2604] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:08.674168 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4129c8-f774-41ce-a7cb-7079df138f41 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:08.675237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4129c8-f774-41ce-a7cb-7079df138f41 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:08.675716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4129c8-f774-41ce-a7cb-7079df138f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:08.709808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-5c4129c8-f774-41ce-a7cb-7079df138f41 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:08.711270 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5c4129c8-f774-41ce-a7cb-7079df138f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:08.721103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1291/2605] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:08.730132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4129c8-f774-41ce-a7cb-7079df138f41 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:08.742021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4129c8-f774-41ce-a7cb-7079df138f41 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:08.748452 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb040320-3adb-4533-8c51-acc3ead61d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:08.749146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb040320-3adb-4533-8c51-acc3ead61d2f admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:08.749427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb040320-3adb-4533-8c51-acc3ead61d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:08.749695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb040320-3adb-4533-8c51-acc3ead61d2f admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:08.750441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb040320-3adb-4533-8c51-acc3ead61d2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:37:08.750902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb040320-3adb-4533-8c51-acc3ead61d2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:08.753921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb040320-3adb-4533-8c51-acc3ead61d2f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:08.771789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1292/2606] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:37:08.781881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb040320-3adb-4533-8c51-acc3ead61d2f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:08.805103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1315/2607] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:08.807110 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318a126-7124-4743-9b07-51948bcd2a19 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:08.807501 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318a126-7124-4743-9b07-51948bcd2a19 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:08.808715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318a126-7124-4743-9b07-51948bcd2a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:08.809197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb040320-3adb-4533-8c51-acc3ead61d2f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:08.817945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8eed4-ca5f-4d89-9505-1c5cae29900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:08.818877 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8eed4-ca5f-4d89-9505-1c5cae29900b admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:08.819336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8eed4-ca5f-4d89-9505-1c5cae29900b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:08.819788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8eed4-ca5f-4d89-9505-1c5cae29900b admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:08.820673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8eed4-ca5f-4d89-9505-1c5cae29900b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:08.824761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8eed4-ca5f-4d89-9505-1c5cae29900b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:08.864409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c318a126-7124-4743-9b07-51948bcd2a19 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:08.866125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c318a126-7124-4743-9b07-51948bcd2a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:08.886083 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1316/2608] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:08.889525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318a126-7124-4743-9b07-51948bcd2a19 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:08.917170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318a126-7124-4743-9b07-51948bcd2a19 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:08.924951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943698b6-7164-4222-9f3b-7efff6a8b922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:08.925755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943698b6-7164-4222-9f3b-7efff6a8b922 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:08.926161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943698b6-7164-4222-9f3b-7efff6a8b922 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:08.926684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943698b6-7164-4222-9f3b-7efff6a8b922 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:08.928188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943698b6-7164-4222-9f3b-7efff6a8b922 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:37:08.928592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943698b6-7164-4222-9f3b-7efff6a8b922 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:08.932420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943698b6-7164-4222-9f3b-7efff6a8b922 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:08.950447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1293/2609] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:08 2022] POST /identity/v3/projects => generated 403 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:08.954510 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8eed4-ca5f-4d89-9505-1c5cae29900b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:08.974136 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1317/2610] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:37:08.981115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943698b6-7164-4222-9f3b-7efff6a8b922 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:08.995441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943698b6-7164-4222-9f3b-7efff6a8b922 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:09.150332 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8eed4-ca5f-4d89-9505-1c5cae29900b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:09.162066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5c82b-9804-474a-b4f5-737322101ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:09.162850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5c82b-9804-474a-b4f5-737322101ced admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:09.163132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5c82b-9804-474a-b4f5-737322101ced admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:09.163402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5c82b-9804-474a-b4f5-737322101ced admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:09.164035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5c82b-9804-474a-b4f5-737322101ced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:09.178451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5c82b-9804-474a-b4f5-737322101ced admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:09.190860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:09.208548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02354b-91ea-4e3e-9834-3ed8b5d62128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:09.209679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02354b-91ea-4e3e-9834-3ed8b5d62128 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:09.210243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02354b-91ea-4e3e-9834-3ed8b5d62128 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:09.210777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02354b-91ea-4e3e-9834-3ed8b5d62128 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:09.211695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02354b-91ea-4e3e-9834-3ed8b5d62128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:09.214156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02354b-91ea-4e3e-9834-3ed8b5d62128 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:09.319115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1294/2611] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:08 2022] POST /identity/v3/users => generated 302 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:09.320317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5c82b-9804-474a-b4f5-737322101ced admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:09.320504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1318/2612] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:08 2022] POST /identity/v3/projects => generated 375 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:37:09.324665 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02354b-91ea-4e3e-9834-3ed8b5d62128 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:09.341466 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5c82b-9804-474a-b4f5-737322101ced admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:09.541051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:09.551605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02354b-91ea-4e3e-9834-3ed8b5d62128 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:09.575124 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8208854-d97b-49af-b742-658a14e74a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:09.575841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208854-d97b-49af-b742-658a14e74a82 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:09.576127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208854-d97b-49af-b742-658a14e74a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:09.576407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208854-d97b-49af-b742-658a14e74a82 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:09.577611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8208854-d97b-49af-b742-658a14e74a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:09.580376 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8208854-d97b-49af-b742-658a14e74a82 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:09.601443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adb1e7-6639-4d07-b485-1ba083376d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:09.602850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adb1e7-6639-4d07-b485-1ba083376d3b admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:09.603302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adb1e7-6639-4d07-b485-1ba083376d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:09.605717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adb1e7-6639-4d07-b485-1ba083376d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:09.606634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adb1e7-6639-4d07-b485-1ba083376d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:09.608633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adb1e7-6639-4d07-b485-1ba083376d3b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:09.625152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1295/2613] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:09.626432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adb1e7-6639-4d07-b485-1ba083376d3b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:09.649860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adb1e7-6639-4d07-b485-1ba083376d3b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:09.663293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2ffb-4a42-4aec-9ae2-e67316df8de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:09.672892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2ffb-4a42-4aec-9ae2-e67316df8de0 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:09.672892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2ffb-4a42-4aec-9ae2-e67316df8de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:09.672892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2ffb-4a42-4aec-9ae2-e67316df8de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:09.672892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f2ffb-4a42-4aec-9ae2-e67316df8de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:09.672892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f2ffb-4a42-4aec-9ae2-e67316df8de0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:09.698897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1296/2614] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:09.698897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c720e-b96e-485a-9d88-8f7938c42817 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:09.698897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c720e-b96e-485a-9d88-8f7938c42817 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:09.698897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c720e-b96e-485a-9d88-8f7938c42817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:09.711886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1319/2615] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:09 2022] POST /identity/v3/users => generated 288 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:09.712763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8208854-d97b-49af-b742-658a14e74a82 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:09.739959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8208854-d97b-49af-b742-658a14e74a82 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:09.750626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-694c720e-b96e-485a-9d88-8f7938c42817 None None] MFA Rules not processed for user `054f2a730eaa4a32b11f4a731c687094`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:09.752130 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-694c720e-b96e-485a-9d88-8f7938c42817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:09.753170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d21bd9-501b-4c1b-89ba-c08d34fa5f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:09.754018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d21bd9-501b-4c1b-89ba-c08d34fa5f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:09.754876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d21bd9-501b-4c1b-89ba-c08d34fa5f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:09.754876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d21bd9-501b-4c1b-89ba-c08d34fa5f48 admin admin] PATH_INFO: `/v3/projects/ebe8f2593c474ff4a6619614e836d89c/users/7f3ad5f301f24df2b7e099486a521e37/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:09.791282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d21bd9-501b-4c1b-89ba-c08d34fa5f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe8f2593c474ff4a6619614e836d89c, user_id=7f3ad5f301f24df2b7e099486a521e37, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:09.794663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d21bd9-501b-4c1b-89ba-c08d34fa5f48 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:09.863564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1320/2616] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:09 2022] PUT /identity/v3/projects/ebe8f2593c474ff4a6619614e836d89c/users/7f3ad5f301f24df2b7e099486a521e37/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:09.865337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d21bd9-501b-4c1b-89ba-c08d34fa5f48 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:09.903949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1297/2617] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:09.903949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771d42c-fa17-4ae0-83e6-3499dd35c398 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:09.903949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771d42c-fa17-4ae0-83e6-3499dd35c398 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:09.903949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771d42c-fa17-4ae0-83e6-3499dd35c398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:09.927484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2771d42c-fa17-4ae0-83e6-3499dd35c398 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:09.929023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2771d42c-fa17-4ae0-83e6-3499dd35c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:10.004260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d21bd9-501b-4c1b-89ba-c08d34fa5f48 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:10.014752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1298/2618] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:10.014752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771d42c-fa17-4ae0-83e6-3499dd35c398 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:10.014752 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2771d42c-fa17-4ae0-83e6-3499dd35c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:10.021352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e13707-0ea3-4e9b-ab6a-3402e2a73467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:10.022322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e13707-0ea3-4e9b-ab6a-3402e2a73467 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:10.022770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e13707-0ea3-4e9b-ab6a-3402e2a73467 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:10.023315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e13707-0ea3-4e9b-ab6a-3402e2a73467 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:10.024137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e13707-0ea3-4e9b-ab6a-3402e2a73467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:10.028555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e13707-0ea3-4e9b-ab6a-3402e2a73467 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:10.045826 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1321/2619] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:10.046848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e13707-0ea3-4e9b-ab6a-3402e2a73467 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:10.050245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-00e13707-0ea3-4e9b-ab6a-3402e2a73467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:10.133517 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771d42c-fa17-4ae0-83e6-3499dd35c398 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:10.140652 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70baf105-9edc-45c4-ab29-b4b200ece2b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:10.141726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70baf105-9edc-45c4-ab29-b4b200ece2b2 service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:10.142409 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70baf105-9edc-45c4-ab29-b4b200ece2b2 service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:10.142807 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e13707-0ea3-4e9b-ab6a-3402e2a73467 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:10.143406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70baf105-9edc-45c4-ab29-b4b200ece2b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:10.151650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edd673-51b9-4655-9fef-a30867e8036a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:10.156739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edd673-51b9-4655-9fef-a30867e8036a service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:10.156739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edd673-51b9-4655-9fef-a30867e8036a service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:10.162173 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edd673-51b9-4655-9fef-a30867e8036a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:10.191270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edd673-51b9-4655-9fef-a30867e8036a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:10.191270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edd673-51b9-4655-9fef-a30867e8036a service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:10.211803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1322/2620] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:10.213398 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edd673-51b9-4655-9fef-a30867e8036a service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:10.250627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70baf105-9edc-45c4-ab29-b4b200ece2b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:10.252174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edd673-51b9-4655-9fef-a30867e8036a service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:10.259696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70baf105-9edc-45c4-ab29-b4b200ece2b2 service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:10.269162 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d40422-4b20-489b-ab54-8939aaaaca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:10.270474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40422-4b20-489b-ab54-8939aaaaca9a admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:10.270872 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40422-4b20-489b-ab54-8939aaaaca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:10.271220 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40422-4b20-489b-ab54-8939aaaaca9a admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:10.271869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d40422-4b20-489b-ab54-8939aaaaca9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:10.272690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1299/2621] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:10.273514 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d40422-4b20-489b-ab54-8939aaaaca9a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:10.294161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1323/2622] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:10.298170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70baf105-9edc-45c4-ab29-b4b200ece2b2 service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:10.311098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70baf105-9edc-45c4-ab29-b4b200ece2b2 service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:10.320990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f193dd-19ae-41cc-91e9-9d0d50e0ef2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:10.321664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f193dd-19ae-41cc-91e9-9d0d50e0ef2b admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:10.321974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f193dd-19ae-41cc-91e9-9d0d50e0ef2b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:10.322247 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f193dd-19ae-41cc-91e9-9d0d50e0ef2b admin admin] PATH_INFO: `/v3/projects/1ca55cca33a941debd3e1fdc8fb3e2a5/users/08966e8cb20d4e0ba14c78b0b72c1336/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:10.347113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f193dd-19ae-41cc-91e9-9d0d50e0ef2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca55cca33a941debd3e1fdc8fb3e2a5, user_id=08966e8cb20d4e0ba14c78b0b72c1336, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:10.348784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f193dd-19ae-41cc-91e9-9d0d50e0ef2b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:10.468221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1300/2623] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:10 2022] PUT /identity/v3/projects/1ca55cca33a941debd3e1fdc8fb3e2a5/users/08966e8cb20d4e0ba14c78b0b72c1336/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:10.478409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b079dbe-ec70-417c-b9b8-336a6fde7ee3 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:10.479104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b079dbe-ec70-417c-b9b8-336a6fde7ee3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:10.479104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b079dbe-ec70-417c-b9b8-336a6fde7ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:10.525291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-7b079dbe-ec70-417c-b9b8-336a6fde7ee3 None None] MFA Rules not processed for user `08966e8cb20d4e0ba14c78b0b72c1336`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:10.526671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-7b079dbe-ec70-417c-b9b8-336a6fde7ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:10.573213 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:10.612782 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1324/2624] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:10.623852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f193dd-19ae-41cc-91e9-9d0d50e0ef2b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:10.626066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f0f193dd-19ae-41cc-91e9-9d0d50e0ef2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:10.719491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f193dd-19ae-41cc-91e9-9d0d50e0ef2b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:10.729571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beca29b-79f2-4cf9-9abb-01a998a4c014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:10.731032 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beca29b-79f2-4cf9-9abb-01a998a4c014 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:10.732302 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beca29b-79f2-4cf9-9abb-01a998a4c014 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:10.733379 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beca29b-79f2-4cf9-9abb-01a998a4c014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:10.804091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beca29b-79f2-4cf9-9abb-01a998a4c014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:10.804091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beca29b-79f2-4cf9-9abb-01a998a4c014 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:10.804091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1301/2625] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:11.008210 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca423c9f-af70-45af-919d-3819bba1da37 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:11.009314 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca423c9f-af70-45af-919d-3819bba1da37 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:11.009825 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca423c9f-af70-45af-919d-3819bba1da37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:11.057099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ca423c9f-af70-45af-919d-3819bba1da37 None None] MFA Rules not processed for user `57662358169d4830b2ebb7d594efb701`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:11.058723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ca423c9f-af70-45af-919d-3819bba1da37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:11.096776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:11.178409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1325/2626] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:11.183062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beca29b-79f2-4cf9-9abb-01a998a4c014 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:11.184822 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-8beca29b-79f2-4cf9-9abb-01a998a4c014 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:11.282260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beca29b-79f2-4cf9-9abb-01a998a4c014 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:11.287279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df45aef-3fab-4bb9-b253-a91a96bfe014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:11.292139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df45aef-3fab-4bb9-b253-a91a96bfe014 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:11.292139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df45aef-3fab-4bb9-b253-a91a96bfe014 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:11.292139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df45aef-3fab-4bb9-b253-a91a96bfe014 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:11.292139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df45aef-3fab-4bb9-b253-a91a96bfe014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:11.292139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df45aef-3fab-4bb9-b253-a91a96bfe014 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:11.379956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1302/2627] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:11 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:11.383841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca423c9f-af70-45af-919d-3819bba1da37 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:11.400028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca423c9f-af70-45af-919d-3819bba1da37 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:11.482492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552698a-ae02-4621-bf5a-5ceec92b6c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:11.483099 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552698a-ae02-4621-bf5a-5ceec92b6c49 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:11.483493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552698a-ae02-4621-bf5a-5ceec92b6c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:11.483994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552698a-ae02-4621-bf5a-5ceec92b6c49 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:11.484636 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552698a-ae02-4621-bf5a-5ceec92b6c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:11.486804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552698a-ae02-4621-bf5a-5ceec92b6c49 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:11.602434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1326/2628] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:11 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:11.606622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df45aef-3fab-4bb9-b253-a91a96bfe014 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:11.617893 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df45aef-3fab-4bb9-b253-a91a96bfe014 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:11.632866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093503d-5780-4f06-9d70-a63cf51c834c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:11.633492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093503d-5780-4f06-9d70-a63cf51c834c admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:11.633774 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093503d-5780-4f06-9d70-a63cf51c834c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:11.634071 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093503d-5780-4f06-9d70-a63cf51c834c admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:11.634817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093503d-5780-4f06-9d70-a63cf51c834c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:11.636431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093503d-5780-4f06-9d70-a63cf51c834c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:11.652329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1303/2629] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:11.656221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552698a-ae02-4621-bf5a-5ceec92b6c49 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:11.670435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552698a-ae02-4621-bf5a-5ceec92b6c49 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:11.684305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ffc92-4ac1-4001-9347-c942ce79c3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:11.685352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ffc92-4ac1-4001-9347-c942ce79c3b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:11.685924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ffc92-4ac1-4001-9347-c942ce79c3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:11.690612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ffc92-4ac1-4001-9347-c942ce79c3b4 admin admin] PATH_INFO: `/v3/projects/3bca2c32913f43ad86326629903c5b4b/users/82b7005da8f04eedab6a32f6dc6484d4/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:11.707477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ffc92-4ac1-4001-9347-c942ce79c3b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bca2c32913f43ad86326629903c5b4b, user_id=82b7005da8f04eedab6a32f6dc6484d4, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:11.710769 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ffc92-4ac1-4001-9347-c942ce79c3b4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:11.811840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1327/2630] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:11 2022] PUT /identity/v3/projects/3bca2c32913f43ad86326629903c5b4b/users/82b7005da8f04eedab6a32f6dc6484d4/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:11.813626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093503d-5780-4f06-9d70-a63cf51c834c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:11.832237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:11.841358 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093503d-5780-4f06-9d70-a63cf51c834c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:11.901960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999b33b-47dd-4386-be13-f54efe5492f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:11.902657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b33b-47dd-4386-be13-f54efe5492f8 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:11.903004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b33b-47dd-4386-be13-f54efe5492f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:11.903337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b33b-47dd-4386-be13-f54efe5492f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:11.904320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999b33b-47dd-4386-be13-f54efe5492f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:11.906004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999b33b-47dd-4386-be13-f54efe5492f8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:11.920771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1304/2631] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:37:11.922921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ffc92-4ac1-4001-9347-c942ce79c3b4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:11.933601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ffc92-4ac1-4001-9347-c942ce79c3b4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:11.938655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa521e1-829d-48c1-af73-944e34adc9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:11.939672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa521e1-829d-48c1-af73-944e34adc9a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:11.940058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa521e1-829d-48c1-af73-944e34adc9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:11.940218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa521e1-829d-48c1-af73-944e34adc9a7 admin admin] PATH_INFO: `/v3/projects/3bca2c32913f43ad86326629903c5b4b/users/82b7005da8f04eedab6a32f6dc6484d4/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:11.942381 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa521e1-829d-48c1-af73-944e34adc9a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bca2c32913f43ad86326629903c5b4b, user_id=82b7005da8f04eedab6a32f6dc6484d4, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:11.944990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa521e1-829d-48c1-af73-944e34adc9a7 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:11.966133 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1328/2632] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:11 2022] PUT /identity/v3/projects/3bca2c32913f43ad86326629903c5b4b/users/82b7005da8f04eedab6a32f6dc6484d4/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:12.075499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1734e6-0e76-42c5-b7c7-502f82a0e466 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:12.075499 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1734e6-0e76-42c5-b7c7-502f82a0e466 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:12.075726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1734e6-0e76-42c5-b7c7-502f82a0e466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:12.124654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-bc1734e6-0e76-42c5-b7c7-502f82a0e466 None None] MFA Rules not processed for user `7f3ad5f301f24df2b7e099486a521e37`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:12.126035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-bc1734e6-0e76-42c5-b7c7-502f82a0e466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:12.197137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1305/2633] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:12 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:12.198030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa521e1-829d-48c1-af73-944e34adc9a7 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:12.323160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:12.329112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa521e1-829d-48c1-af73-944e34adc9a7 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:12.349921 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732ff6a-b2d5-4f12-ba40-c36d3c72082a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:12.350859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732ff6a-b2d5-4f12-ba40-c36d3c72082a admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:12.351442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732ff6a-b2d5-4f12-ba40-c36d3c72082a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:12.351932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732ff6a-b2d5-4f12-ba40-c36d3c72082a admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:12.353065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732ff6a-b2d5-4f12-ba40-c36d3c72082a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:12.356616 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732ff6a-b2d5-4f12-ba40-c36d3c72082a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:12.481456 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1329/2634] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:12 2022] POST /identity/v3/projects => generated 403 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:37:12.485856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1734e6-0e76-42c5-b7c7-502f82a0e466 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:12.500100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1734e6-0e76-42c5-b7c7-502f82a0e466 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:12.599957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a340ed-d337-45cb-afda-7b836f23372d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:12.600693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a340ed-d337-45cb-afda-7b836f23372d admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:12.601072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a340ed-d337-45cb-afda-7b836f23372d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:12.601442 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a340ed-d337-45cb-afda-7b836f23372d admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:12.602237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a340ed-d337-45cb-afda-7b836f23372d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:12.604490 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a340ed-d337-45cb-afda-7b836f23372d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:12.699831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1306/2635] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:12 2022] POST /identity/v3/users => generated 308 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:12.700031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732ff6a-b2d5-4f12-ba40-c36d3c72082a admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:12.710539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732ff6a-b2d5-4f12-ba40-c36d3c72082a admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:12.715542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b1ad0-59f8-445d-897a-fbc7a5c77b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:12.716443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b1ad0-59f8-445d-897a-fbc7a5c77b8d admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:12.716995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b1ad0-59f8-445d-897a-fbc7a5c77b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:12.717363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b1ad0-59f8-445d-897a-fbc7a5c77b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:12.718181 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b1ad0-59f8-445d-897a-fbc7a5c77b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:12.720222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b1ad0-59f8-445d-897a-fbc7a5c77b8d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:12.743470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1330/2636] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:37:12.747419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a340ed-d337-45cb-afda-7b836f23372d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:12.761536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a340ed-d337-45cb-afda-7b836f23372d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:12.771035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b14c-282e-464c-89b6-4869dafba51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:12.771914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b14c-282e-464c-89b6-4869dafba51a admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:12.772286 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b14c-282e-464c-89b6-4869dafba51a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:12.772676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b14c-282e-464c-89b6-4869dafba51a admin admin] PATH_INFO: `/v3/projects/d75f1226db0646e2b1e649517b1bde2a/users/349440685e864dd5a3311119a772c533/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:12.824355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318b14c-282e-464c-89b6-4869dafba51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75f1226db0646e2b1e649517b1bde2a, user_id=349440685e864dd5a3311119a772c533, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:12.829705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318b14c-282e-464c-89b6-4869dafba51a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:12.956505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1307/2637] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:12 2022] PUT /identity/v3/projects/d75f1226db0646e2b1e649517b1bde2a/users/349440685e864dd5a3311119a772c533/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:12.958402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b1ad0-59f8-445d-897a-fbc7a5c77b8d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:12.976092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b1ad0-59f8-445d-897a-fbc7a5c77b8d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:13.081179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874c03d-2a8a-42c8-9374-d4e76857d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:13.083609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874c03d-2a8a-42c8-9374-d4e76857d6af admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:13.083899 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874c03d-2a8a-42c8-9374-d4e76857d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:13.084171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874c03d-2a8a-42c8-9374-d4e76857d6af admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:13.086300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874c03d-2a8a-42c8-9374-d4e76857d6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:13.088119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874c03d-2a8a-42c8-9374-d4e76857d6af admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:13.102950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1331/2638] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:37:13.105777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b14c-282e-464c-89b6-4869dafba51a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:13.118691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b14c-282e-464c-89b6-4869dafba51a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:13.124661 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2f2dd-c332-4599-8ae1-4daffdb613cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:13.125322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2f2dd-c332-4599-8ae1-4daffdb613cf admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:13.125599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2f2dd-c332-4599-8ae1-4daffdb613cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:13.125900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2f2dd-c332-4599-8ae1-4daffdb613cf admin admin] PATH_INFO: `/v3/projects/d75f1226db0646e2b1e649517b1bde2a/users/349440685e864dd5a3311119a772c533/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:13.127887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2f2dd-c332-4599-8ae1-4daffdb613cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75f1226db0646e2b1e649517b1bde2a, user_id=349440685e864dd5a3311119a772c533, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:13.132149 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2f2dd-c332-4599-8ae1-4daffdb613cf admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:13.150443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1308/2639] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:13 2022] PUT /identity/v3/projects/d75f1226db0646e2b1e649517b1bde2a/users/349440685e864dd5a3311119a772c533/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:13.608761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:13.868813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46baa4-a80d-417f-8c0e-bc6147516bd8 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:13.869389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46baa4-a80d-417f-8c0e-bc6147516bd8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:13.869389 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46baa4-a80d-417f-8c0e-bc6147516bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:13.890852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ef46baa4-a80d-417f-8c0e-bc6147516bd8 None None] MFA Rules not processed for user `054f2a730eaa4a32b11f4a731c687094`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:13.892064 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ef46baa4-a80d-417f-8c0e-bc6147516bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:13.964752 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1332/2640] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:14.588015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:15.453112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fea0f-02a7-4ccc-9798-ae81ddc1d7c7 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:15.454061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fea0f-02a7-4ccc-9798-ae81ddc1d7c7 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:15.454576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fea0f-02a7-4ccc-9798-ae81ddc1d7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:15.487629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-677fea0f-02a7-4ccc-9798-ae81ddc1d7c7 None None] MFA Rules not processed for user `349440685e864dd5a3311119a772c533`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:15.489447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-677fea0f-02a7-4ccc-9798-ae81ddc1d7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:15.686838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1309/2641] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:15 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:16.177606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46baa4-a80d-417f-8c0e-bc6147516bd8 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:16.259005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46baa4-a80d-417f-8c0e-bc6147516bd8 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:16.267587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:16.269060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:16.269510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:16.270315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] PATH_INFO: `/v3/users/08966e8cb20d4e0ba14c78b0b72c1336` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:16.273144 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08966e8cb20d4e0ba14c78b0b72c1336)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:16.275436 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:16.511427 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08966e8cb20d4e0ba14c78b0b72c1336'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:16.669873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08966e8cb20d4e0ba14c78b0b72c1336'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:16.693978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08966e8cb20d4e0ba14c78b0b72c1336'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:16.731455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08966e8cb20d4e0ba14c78b0b72c1336'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:16.738296 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08966e8cb20d4e0ba14c78b0b72c1336'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:16.750151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1333/2642] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:37:16 2022] DELETE /identity/v3/users/08966e8cb20d4e0ba14c78b0b72c1336 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:16.755480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf1a05-f02a-460c-a71a-76032d111613 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:16.755990 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf1a05-f02a-460c-a71a-76032d111613 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:16.756114 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf1a05-f02a-460c-a71a-76032d111613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:16.781357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-21cf1a05-f02a-460c-a71a-76032d111613 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:16.790641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-21cf1a05-f02a-460c-a71a-76032d111613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:16.846964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:16.960153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1310/2643] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:16.980932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:16.982487 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:17.111411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043c3b-0fdf-4eb9-b0ac-7f7e9bfd2ca1 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:17.126062 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01facc-5b94-49cd-9146-cc744fbf944b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:17.126933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01facc-5b94-49cd-9146-cc744fbf944b service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:17.127317 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01facc-5b94-49cd-9146-cc744fbf944b service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:17.128388 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01facc-5b94-49cd-9146-cc744fbf944b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:17.143859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01facc-5b94-49cd-9146-cc744fbf944b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:17.146114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01facc-5b94-49cd-9146-cc744fbf944b service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:17.161950 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:17.173014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1334/2644] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:18.037325 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf1a05-f02a-460c-a71a-76032d111613 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:18.040199 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-21cf1a05-f02a-460c-a71a-76032d111613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:18.061432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf1a05-f02a-460c-a71a-76032d111613 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:18.069762 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:18.071237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:18.071706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:18.072514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] PATH_INFO: `/v3/users/7f3ad5f301f24df2b7e099486a521e37` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:18.074691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f3ad5f301f24df2b7e099486a521e37)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:18.078756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:18.271467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01facc-5b94-49cd-9146-cc744fbf944b service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:18.273031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2a01facc-5b94-49cd-9146-cc744fbf944b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:18.291844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01facc-5b94-49cd-9146-cc744fbf944b service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:18.296708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:18.297446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:18.297894 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:18.298253 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] PATH_INFO: `/v3/projects/1ca55cca33a941debd3e1fdc8fb3e2a5` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:18.299492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca55cca33a941debd3e1fdc8fb3e2a5)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:18.301704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:18.321295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f3ad5f301f24df2b7e099486a521e37'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.435755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f3ad5f301f24df2b7e099486a521e37'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.456756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f3ad5f301f24df2b7e099486a521e37'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.468458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f3ad5f301f24df2b7e099486a521e37'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.468658 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca55cca33a941debd3e1fdc8fb3e2a5', 'request_id': 'req-d5d24349-4868-46b7-9b2d-79616c810ee8'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.470038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ffa2d8ee-a02b-4d70-afa3-c86dbd2ff17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f3ad5f301f24df2b7e099486a521e37'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.477477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca55cca33a941debd3e1fdc8fb3e2a5', 'request_id': 'req-d5d24349-4868-46b7-9b2d-79616c810ee8'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.478983 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1311/2645] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:37:18 2022] DELETE /identity/v3/users/7f3ad5f301f24df2b7e099486a521e37 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:18.610988 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca55cca33a941debd3e1fdc8fb3e2a5', 'request_id': 'req-d5d24349-4868-46b7-9b2d-79616c810ee8'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.626815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] The token cache is being invalidate because project 1ca55cca33a941debd3e1fdc8fb3e2a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:37:18.627416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.629164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d5d24349-4868-46b7-9b2d-79616c810ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:18.634827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1335/2646] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:37:18 2022] DELETE /identity/v3/projects/1ca55cca33a941debd3e1fdc8fb3e2a5 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:18.636537 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c9c7f-4922-40d4-a28e-653dfa1641ef None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:18.636921 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c9c7f-4922-40d4-a28e-653dfa1641ef None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:18.637309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c9c7f-4922-40d4-a28e-653dfa1641ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:18.682976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-f27c9c7f-4922-40d4-a28e-653dfa1641ef None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:18.684230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f27c9c7f-4922-40d4-a28e-653dfa1641ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:18.800344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1312/2647] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:18.808735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb112dd5-443e-48c5-8629-fedfb3195ace None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:18.809053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb112dd5-443e-48c5-8629-fedfb3195ace None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:18.809312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb112dd5-443e-48c5-8629-fedfb3195ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:18.847329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-cb112dd5-443e-48c5-8629-fedfb3195ace None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:18.848246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c9c7f-4922-40d4-a28e-653dfa1641ef None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:18.848984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cb112dd5-443e-48c5-8629-fedfb3195ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:18.850601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f27c9c7f-4922-40d4-a28e-653dfa1641ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:18.862661 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1336/2648] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:18.863714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb112dd5-443e-48c5-8629-fedfb3195ace None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:18.881744 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb112dd5-443e-48c5-8629-fedfb3195ace None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:18.883123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408d8c0-154c-4e4a-969d-5e7fd3983074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:18.883844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408d8c0-154c-4e4a-969d-5e7fd3983074 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:18.884190 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408d8c0-154c-4e4a-969d-5e7fd3983074 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:18.884610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408d8c0-154c-4e4a-969d-5e7fd3983074 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:18.885340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408d8c0-154c-4e4a-969d-5e7fd3983074 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:37:18.885711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408d8c0-154c-4e4a-969d-5e7fd3983074 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:18.887700 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408d8c0-154c-4e4a-969d-5e7fd3983074 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:18.897549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c9c7f-4922-40d4-a28e-653dfa1641ef None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:18.903980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1337/2649] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:37:18.909407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408d8c0-154c-4e4a-969d-5e7fd3983074 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:18.912060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:18.912789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:18.913141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:18.913525 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] PATH_INFO: `/v3/projects/ebe8f2593c474ff4a6619614e836d89c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:18.916316 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebe8f2593c474ff4a6619614e836d89c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:18.918523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:18.926063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408d8c0-154c-4e4a-969d-5e7fd3983074 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:18.942702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81077673-19ea-469e-b8a3-78e3c78126ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:18.943473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81077673-19ea-469e-b8a3-78e3c78126ea admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:18.943877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81077673-19ea-469e-b8a3-78e3c78126ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:18.944249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81077673-19ea-469e-b8a3-78e3c78126ea admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:18.944979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81077673-19ea-469e-b8a3-78e3c78126ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:18.947708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81077673-19ea-469e-b8a3-78e3c78126ea admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:19.063949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:19.077558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe8f2593c474ff4a6619614e836d89c', 'request_id': 'req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.086826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe8f2593c474ff4a6619614e836d89c', 'request_id': 'req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.151941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1338/2650] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:18 2022] POST /identity/v3/projects => generated 373 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:19.153913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81077673-19ea-469e-b8a3-78e3c78126ea admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:19.167340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe8f2593c474ff4a6619614e836d89c', 'request_id': 'req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.185067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] The token cache is being invalidate because project ebe8f2593c474ff4a6619614e836d89c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:37:19.185776 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.187038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.193695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1313/2651] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:37:18 2022] DELETE /identity/v3/projects/ebe8f2593c474ff4a6619614e836d89c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:19.194872 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:19.198416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:19.262989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81077673-19ea-469e-b8a3-78e3c78126ea admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:19.271112 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d800027-b204-45be-9c4e-af53046f7a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:19.272405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d800027-b204-45be-9c4e-af53046f7a8f admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:19.273354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d800027-b204-45be-9c4e-af53046f7a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:19.273949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d800027-b204-45be-9c4e-af53046f7a8f admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:19.275617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d800027-b204-45be-9c4e-af53046f7a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:19.278591 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d800027-b204-45be-9c4e-af53046f7a8f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:19.328125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae7b3f-a3c1-44a5-a54f-02acab333ae0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:19.334599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:19.335639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:19.336210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:19.337002 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] PATH_INFO: `/v3/users/349440685e864dd5a3311119a772c533` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:19.338630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=349440685e864dd5a3311119a772c533)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:19.341353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:19.384168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1339/2652] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:19 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:19.385559 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d800027-b204-45be-9c4e-af53046f7a8f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:19.387735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2d800027-b204-45be-9c4e-af53046f7a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:19.402917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d800027-b204-45be-9c4e-af53046f7a8f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:19.409637 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9dc74-20ef-4ec0-a828-4a957d1175c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:19.411084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9dc74-20ef-4ec0-a828-4a957d1175c7 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:19.411891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9dc74-20ef-4ec0-a828-4a957d1175c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:19.411891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9dc74-20ef-4ec0-a828-4a957d1175c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:19.412260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9dc74-20ef-4ec0-a828-4a957d1175c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:19.413915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9dc74-20ef-4ec0-a828-4a957d1175c7 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:19.432955 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1340/2653] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:19.433170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9dc74-20ef-4ec0-a828-4a957d1175c7 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:19.445728 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9dc74-20ef-4ec0-a828-4a957d1175c7 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:19.453969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fccd5-e1e2-43ee-81a8-d2715e5cd79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:19.454603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fccd5-e1e2-43ee-81a8-d2715e5cd79f admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:19.462391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fccd5-e1e2-43ee-81a8-d2715e5cd79f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:19.462391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fccd5-e1e2-43ee-81a8-d2715e5cd79f admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:19.462391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fccd5-e1e2-43ee-81a8-d2715e5cd79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:19.462391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fccd5-e1e2-43ee-81a8-d2715e5cd79f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:19.469382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1341/2654] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:19.476564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fccd5-e1e2-43ee-81a8-d2715e5cd79f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:19.489739 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fccd5-e1e2-43ee-81a8-d2715e5cd79f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:19.494738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb917d86-3cc7-4a08-a6bd-86c129ce73e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:19.495586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917d86-3cc7-4a08-a6bd-86c129ce73e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:19.497642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917d86-3cc7-4a08-a6bd-86c129ce73e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:19.497958 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917d86-3cc7-4a08-a6bd-86c129ce73e7 admin admin] PATH_INFO: `/v3/projects/13bc1b020223465391d6c06a4881deda/users/e36f881f776842a8a9ba60108fe6e1f8/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:19.522198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb917d86-3cc7-4a08-a6bd-86c129ce73e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bc1b020223465391d6c06a4881deda, user_id=e36f881f776842a8a9ba60108fe6e1f8, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:19.524981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb917d86-3cc7-4a08-a6bd-86c129ce73e7 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:19.634687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1342/2655] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:19 2022] PUT /identity/v3/projects/13bc1b020223465391d6c06a4881deda/users/e36f881f776842a8a9ba60108fe6e1f8/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:19.636192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342db534-a4e3-4ab6-8a14-c0a369af19a8 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:19.636527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342db534-a4e3-4ab6-8a14-c0a369af19a8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:19.636798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342db534-a4e3-4ab6-8a14-c0a369af19a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:19.641040 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349440685e864dd5a3311119a772c533'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.671480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-342db534-a4e3-4ab6-8a14-c0a369af19a8 None None] MFA Rules not processed for user `e36f881f776842a8a9ba60108fe6e1f8`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:19.672949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-342db534-a4e3-4ab6-8a14-c0a369af19a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:19.745199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1343/2656] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:19.753914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349440685e864dd5a3311119a772c533'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.762863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342db534-a4e3-4ab6-8a14-c0a369af19a8 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:19.764830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-342db534-a4e3-4ab6-8a14-c0a369af19a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:19.776915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349440685e864dd5a3311119a772c533'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.787424 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349440685e864dd5a3311119a772c533'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.788233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349440685e864dd5a3311119a772c533'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:19.794228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1314/2657] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:37:19 2022] DELETE /identity/v3/users/349440685e864dd5a3311119a772c533 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:19.836369 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342db534-a4e3-4ab6-8a14-c0a369af19a8 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:19.891259 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:19.914984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:19.915911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:19.916382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:19.917080 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:19.919938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.026225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:20.027754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:20.038261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1344/2658] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:37:20.072183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:20.073679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.094011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:20.096935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.193129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c289e-a17a-4648-b9b6-505335e2ad35 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:20.194558 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:20.195234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:20.195506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:20.195771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] PATH_INFO: `/v3/users/e36f881f776842a8a9ba60108fe6e1f8` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:20.196949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e36f881f776842a8a9ba60108fe6e1f8)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:20.198830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:20.219520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f3752-188d-424c-b13c-425648c7091a service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:20.225765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:20.227551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:20.227551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:20.228555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] PATH_INFO: `/v3/projects/d75f1226db0646e2b1e649517b1bde2a` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:20.230400 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75f1226db0646e2b1e649517b1bde2a)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:20.231904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:20.414706 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e36f881f776842a8a9ba60108fe6e1f8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.418782 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75f1226db0646e2b1e649517b1bde2a', 'request_id': 'req-95f9f09c-da35-4906-842f-b1b1b780f26e'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.427152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75f1226db0646e2b1e649517b1bde2a', 'request_id': 'req-95f9f09c-da35-4906-842f-b1b1b780f26e'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.491680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e36f881f776842a8a9ba60108fe6e1f8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.509254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e36f881f776842a8a9ba60108fe6e1f8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.517694 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e36f881f776842a8a9ba60108fe6e1f8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.518502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3ff483ae-bfd6-4426-886a-269ba4d29aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e36f881f776842a8a9ba60108fe6e1f8'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.525050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1315/2659] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:37:20 2022] DELETE /identity/v3/users/e36f881f776842a8a9ba60108fe6e1f8 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:20.529611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673e209-b47a-4632-a45c-647c92d62074 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:20.530206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673e209-b47a-4632-a45c-647c92d62074 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:20.531398 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673e209-b47a-4632-a45c-647c92d62074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:20.554627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9673e209-b47a-4632-a45c-647c92d62074 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:20.555850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9673e209-b47a-4632-a45c-647c92d62074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.617908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1316/2660] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:20.627641 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673e209-b47a-4632-a45c-647c92d62074 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:20.629416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9673e209-b47a-4632-a45c-647c92d62074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.629841 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75f1226db0646e2b1e649517b1bde2a', 'request_id': 'req-95f9f09c-da35-4906-842f-b1b1b780f26e'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.642434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] The token cache is being invalidate because project d75f1226db0646e2b1e649517b1bde2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:37:20.643328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.644353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95f9f09c-da35-4906-842f-b1b1b780f26e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:20.646599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1345/2661] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:37:20 2022] DELETE /identity/v3/projects/d75f1226db0646e2b1e649517b1bde2a => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:20.651500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc3c90-3e68-450e-aa0c-ebd22d01f379 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:20.651882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc3c90-3e68-450e-aa0c-ebd22d01f379 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:20.652465 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc3c90-3e68-450e-aa0c-ebd22d01f379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:20.674626 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3fdc3c90-3e68-450e-aa0c-ebd22d01f379 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:20.676152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3fdc3c90-3e68-450e-aa0c-ebd22d01f379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.685527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1346/2662] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:20.694268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d923a-8c82-471e-a9c8-366745535424 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:20.703645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d923a-8c82-471e-a9c8-366745535424 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:20.703645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d923a-8c82-471e-a9c8-366745535424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:20.718201 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-cd6d923a-8c82-471e-a9c8-366745535424 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:20.719548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cd6d923a-8c82-471e-a9c8-366745535424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.725072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673e209-b47a-4632-a45c-647c92d62074 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:20.731267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1347/2663] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:20.731267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef155692-c60d-4d3b-b611-18b1dfbb7b52 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:20.731267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef155692-c60d-4d3b-b611-18b1dfbb7b52 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:20.731267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef155692-c60d-4d3b-b611-18b1dfbb7b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:20.735966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86742fc2-9b37-44c0-a864-fd43e2012cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:20.736769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86742fc2-9b37-44c0-a864-fd43e2012cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:20.737203 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86742fc2-9b37-44c0-a864-fd43e2012cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:20.737475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86742fc2-9b37-44c0-a864-fd43e2012cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:20.740267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-86742fc2-9b37-44c0-a864-fd43e2012cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.755067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86742fc2-9b37-44c0-a864-fd43e2012cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:20.756553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86742fc2-9b37-44c0-a864-fd43e2012cc6 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:20.766287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-ef155692-c60d-4d3b-b611-18b1dfbb7b52 None None] MFA Rules not processed for user `82b7005da8f04eedab6a32f6dc6484d4`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:20.767726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1317/2664] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:37:20.767897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ef155692-c60d-4d3b-b611-18b1dfbb7b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.769099 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86742fc2-9b37-44c0-a864-fd43e2012cc6 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:20.789679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86742fc2-9b37-44c0-a864-fd43e2012cc6 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:20.795164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0295cce-345d-4102-ad1d-24655738ec13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:20.795531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0295cce-345d-4102-ad1d-24655738ec13 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:20.795819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0295cce-345d-4102-ad1d-24655738ec13 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:20.796106 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0295cce-345d-4102-ad1d-24655738ec13 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:20.796943 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0295cce-345d-4102-ad1d-24655738ec13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:37:20.798367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0295cce-345d-4102-ad1d-24655738ec13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:20.801122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0295cce-345d-4102-ad1d-24655738ec13 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:20.821583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1318/2665] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:37:20.823209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0295cce-345d-4102-ad1d-24655738ec13 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:20.846472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0295cce-345d-4102-ad1d-24655738ec13 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:20.849771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da2cdd-ca43-4696-a7ec-cbef47ac085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:20.850574 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da2cdd-ca43-4696-a7ec-cbef47ac085d admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:20.850864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da2cdd-ca43-4696-a7ec-cbef47ac085d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:20.851343 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da2cdd-ca43-4696-a7ec-cbef47ac085d admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:20.852196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da2cdd-ca43-4696-a7ec-cbef47ac085d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:20.862756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da2cdd-ca43-4696-a7ec-cbef47ac085d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:20.905848 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1348/2666] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:20.921845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef155692-c60d-4d3b-b611-18b1dfbb7b52 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:20.925928 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ef155692-c60d-4d3b-b611-18b1dfbb7b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:20.963750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1319/2667] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:20 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:37:20.968587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da2cdd-ca43-4696-a7ec-cbef47ac085d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:21.051166 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef155692-c60d-4d3b-b611-18b1dfbb7b52 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:21.108704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da2cdd-ca43-4696-a7ec-cbef47ac085d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:21.124461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88733808-c8db-4085-8796-7e707b2fa734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:21.125076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88733808-c8db-4085-8796-7e707b2fa734 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:21.125403 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88733808-c8db-4085-8796-7e707b2fa734 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:21.125674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88733808-c8db-4085-8796-7e707b2fa734 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:21.126825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88733808-c8db-4085-8796-7e707b2fa734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:21.129023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88733808-c8db-4085-8796-7e707b2fa734 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:21.137970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92d765-eb26-4afa-bcdd-0699193821c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:21.138815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92d765-eb26-4afa-bcdd-0699193821c8 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:21.138962 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92d765-eb26-4afa-bcdd-0699193821c8 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:21.139329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92d765-eb26-4afa-bcdd-0699193821c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:21.140695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:21.142117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4a92d765-eb26-4afa-bcdd-0699193821c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:21.225558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92d765-eb26-4afa-bcdd-0699193821c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:21.228335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92d765-eb26-4afa-bcdd-0699193821c8 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:21.235420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1320/2668] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:20 2022] POST /identity/v3/users => generated 287 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:21.236164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88733808-c8db-4085-8796-7e707b2fa734 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:21.251374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1349/2669] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:37:21.252372 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88733808-c8db-4085-8796-7e707b2fa734 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:21.264261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0421b8-a97b-468f-b896-15aeee4aaf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:21.264261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0421b8-a97b-468f-b896-15aeee4aaf11 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:21.264701 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0421b8-a97b-468f-b896-15aeee4aaf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:21.264885 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0421b8-a97b-468f-b896-15aeee4aaf11 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:21.265509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0421b8-a97b-468f-b896-15aeee4aaf11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:21.270527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0421b8-a97b-468f-b896-15aeee4aaf11 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:21.292556 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1321/2670] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:37:21.296137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92d765-eb26-4afa-bcdd-0699193821c8 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:21.313294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92d765-eb26-4afa-bcdd-0699193821c8 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:21.318951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b33e0-d6e2-4bcc-9477-00c368648238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:21.319669 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b33e0-d6e2-4bcc-9477-00c368648238 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:21.319852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b33e0-d6e2-4bcc-9477-00c368648238 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:21.320121 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b33e0-d6e2-4bcc-9477-00c368648238 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:21.320729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b33e0-d6e2-4bcc-9477-00c368648238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:21.322493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b33e0-d6e2-4bcc-9477-00c368648238 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:21.336842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1350/2671] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:21.339731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0421b8-a97b-468f-b896-15aeee4aaf11 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:21.351954 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0421b8-a97b-468f-b896-15aeee4aaf11 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:21.358444 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2a90f-7938-4f9a-88e0-cd2b49a80983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:21.360089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2a90f-7938-4f9a-88e0-cd2b49a80983 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:21.360261 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2a90f-7938-4f9a-88e0-cd2b49a80983 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:21.369754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2a90f-7938-4f9a-88e0-cd2b49a80983 admin admin] PATH_INFO: `/v3/projects/dbe2f85ec90a49f4940d8d3067feb7cb/users/8c37ba4362ea49fa9eacbfb2e98dc581/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:21.382726 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2a90f-7938-4f9a-88e0-cd2b49a80983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe2f85ec90a49f4940d8d3067feb7cb, user_id=8c37ba4362ea49fa9eacbfb2e98dc581, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:21.385463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2a90f-7938-4f9a-88e0-cd2b49a80983 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:21.538745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1322/2672] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:21 2022] PUT /identity/v3/projects/dbe2f85ec90a49f4940d8d3067feb7cb/users/8c37ba4362ea49fa9eacbfb2e98dc581/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:21.539471 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d51bf-eaf5-4cb8-8c0c-df2c57cda1df None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:21.539934 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d51bf-eaf5-4cb8-8c0c-df2c57cda1df None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:21.540040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d51bf-eaf5-4cb8-8c0c-df2c57cda1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:21.578483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1c1d51bf-eaf5-4cb8-8c0c-df2c57cda1df None None] MFA Rules not processed for user `8c37ba4362ea49fa9eacbfb2e98dc581`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:21.581472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1c1d51bf-eaf5-4cb8-8c0c-df2c57cda1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:21.669813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2a90f-7938-4f9a-88e0-cd2b49a80983 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:21.671576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-08c2a90f-7938-4f9a-88e0-cd2b49a80983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:21.716384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1351/2673] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:21.720593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d51bf-eaf5-4cb8-8c0c-df2c57cda1df None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:21.838893 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d51bf-eaf5-4cb8-8c0c-df2c57cda1df None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:21.846223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e172-ba6b-49cd-834f-ac0a04799b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:21.847350 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e172-ba6b-49cd-834f-ac0a04799b5d admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:21.847989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e172-ba6b-49cd-834f-ac0a04799b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:21.848830 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e172-ba6b-49cd-834f-ac0a04799b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:21.849723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72e172-ba6b-49cd-834f-ac0a04799b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:21.851923 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72e172-ba6b-49cd-834f-ac0a04799b5d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:21.877939 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2a90f-7938-4f9a-88e0-cd2b49a80983 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:21.883857 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:21.884644 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:21.884988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:21.885330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] PATH_INFO: `/v3/projects/13bc1b020223465391d6c06a4881deda` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:21.886587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13bc1b020223465391d6c06a4881deda)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:21.888566 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:21.893232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:21.968794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1352/2674] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:21 2022] POST /identity/v3/projects => generated 373 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:21.973214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e172-ba6b-49cd-834f-ac0a04799b5d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.039323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e172-ba6b-49cd-834f-ac0a04799b5d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.052049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13bc1b020223465391d6c06a4881deda', 'request_id': 'req-f403973c-5cfa-4bb3-9825-2adcebad3b98'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:22.063117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13bc1b020223465391d6c06a4881deda', 'request_id': 'req-f403973c-5cfa-4bb3-9825-2adcebad3b98'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:22.106516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eff425-7b70-47c6-94ed-1cc37dd599fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.107188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eff425-7b70-47c6-94ed-1cc37dd599fa admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.107465 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eff425-7b70-47c6-94ed-1cc37dd599fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.107836 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eff425-7b70-47c6-94ed-1cc37dd599fa admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.108355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eff425-7b70-47c6-94ed-1cc37dd599fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.114551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eff425-7b70-47c6-94ed-1cc37dd599fa admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.155704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13bc1b020223465391d6c06a4881deda', 'request_id': 'req-f403973c-5cfa-4bb3-9825-2adcebad3b98'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:22.167864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] The token cache is being invalidate because project 13bc1b020223465391d6c06a4881deda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:37:22.168150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:22.168985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-f403973c-5cfa-4bb3-9825-2adcebad3b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:37:22.170536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1323/2675] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:37:21 2022] DELETE /identity/v3/projects/13bc1b020223465391d6c06a4881deda => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:22.175976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a35b2-967f-41b1-9dd3-54c056be282d None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.176254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a35b2-967f-41b1-9dd3-54c056be282d None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.176516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a35b2-967f-41b1-9dd3-54c056be282d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.190724 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1353/2676] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:21 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:37:22.197972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-dc3a35b2-967f-41b1-9dd3-54c056be282d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:22.199434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-dc3a35b2-967f-41b1-9dd3-54c056be282d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:22.201687 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eff425-7b70-47c6-94ed-1cc37dd599fa admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.203460 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-88eff425-7b70-47c6-94ed-1cc37dd599fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:22.209679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1324/2677] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:22.217300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9cf51-0259-43a7-80f3-98f1fe0a648c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.217542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eff425-7b70-47c6-94ed-1cc37dd599fa admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.218045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9cf51-0259-43a7-80f3-98f1fe0a648c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.218489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9cf51-0259-43a7-80f3-98f1fe0a648c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.223047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019490bf-62ec-4071-bc6b-2daf196f2aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.223930 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019490bf-62ec-4071-bc6b-2daf196f2aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.224204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019490bf-62ec-4071-bc6b-2daf196f2aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.228680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019490bf-62ec-4071-bc6b-2daf196f2aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.228680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019490bf-62ec-4071-bc6b-2daf196f2aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.228680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019490bf-62ec-4071-bc6b-2daf196f2aa1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.242214 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1354/2678] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:22.244150 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019490bf-62ec-4071-bc6b-2daf196f2aa1 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.245619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b4b9cf51-0259-43a7-80f3-98f1fe0a648c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:22.247102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b4b9cf51-0259-43a7-80f3-98f1fe0a648c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:22.255140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019490bf-62ec-4071-bc6b-2daf196f2aa1 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.261031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1325/2679] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:22.262129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9cf51-0259-43a7-80f3-98f1fe0a648c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.264216 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661232d-a38c-45db-9a02-2fa340298935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.265039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661232d-a38c-45db-9a02-2fa340298935 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.265439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661232d-a38c-45db-9a02-2fa340298935 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.265840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661232d-a38c-45db-9a02-2fa340298935 admin admin] PATH_INFO: `/v3/projects/4996d609438740f79b07cec536b9681b/users/91949ba2b46140bab0bbbc465cc26b37/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.272749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9cf51-0259-43a7-80f3-98f1fe0a648c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.279194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ed7dc-581e-4d82-aa01-8a28c458583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.279194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ed7dc-581e-4d82-aa01-8a28c458583e admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.279194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ed7dc-581e-4d82-aa01-8a28c458583e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.279512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ed7dc-581e-4d82-aa01-8a28c458583e admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.280226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ed7dc-581e-4d82-aa01-8a28c458583e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:37:22.280528 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ed7dc-581e-4d82-aa01-8a28c458583e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.282618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ed7dc-581e-4d82-aa01-8a28c458583e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.287149 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661232d-a38c-45db-9a02-2fa340298935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4996d609438740f79b07cec536b9681b, user_id=91949ba2b46140bab0bbbc465cc26b37, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.290089 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661232d-a38c-45db-9a02-2fa340298935 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.296237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1326/2680] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:37:22.300912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ed7dc-581e-4d82-aa01-8a28c458583e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.311501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ed7dc-581e-4d82-aa01-8a28c458583e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.316386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdaf73-13b3-4b98-8a61-9ff029fd0605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.317000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdaf73-13b3-4b98-8a61-9ff029fd0605 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.317279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdaf73-13b3-4b98-8a61-9ff029fd0605 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.317555 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdaf73-13b3-4b98-8a61-9ff029fd0605 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.318229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdaf73-13b3-4b98-8a61-9ff029fd0605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.320235 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdaf73-13b3-4b98-8a61-9ff029fd0605 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.377328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1355/2681] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:22 2022] PUT /identity/v3/projects/4996d609438740f79b07cec536b9681b/users/91949ba2b46140bab0bbbc465cc26b37/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:22.381334 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661232d-a38c-45db-9a02-2fa340298935 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.450065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661232d-a38c-45db-9a02-2fa340298935 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.454946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb259108-1bb0-4ab1-b2fb-240b646545c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.455391 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1327/2682] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:22 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:37:22.456714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb259108-1bb0-4ab1-b2fb-240b646545c7 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.457026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb259108-1bb0-4ab1-b2fb-240b646545c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.457281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb259108-1bb0-4ab1-b2fb-240b646545c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.457973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb259108-1bb0-4ab1-b2fb-240b646545c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.459716 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdaf73-13b3-4b98-8a61-9ff029fd0605 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.463339 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb259108-1bb0-4ab1-b2fb-240b646545c7 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.486435 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1356/2683] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:22.488093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb259108-1bb0-4ab1-b2fb-240b646545c7 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.500876 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb259108-1bb0-4ab1-b2fb-240b646545c7 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.535194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdaf73-13b3-4b98-8a61-9ff029fd0605 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.541974 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088e021-484e-44ee-b219-45913996ec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.542861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088e021-484e-44ee-b219-45913996ec5e admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.554406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088e021-484e-44ee-b219-45913996ec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.554406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088e021-484e-44ee-b219-45913996ec5e admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.554406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088e021-484e-44ee-b219-45913996ec5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.554406 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088e021-484e-44ee-b219-45913996ec5e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.568800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5a955-cd3d-48d2-b036-75b1c63f4803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.569570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5a955-cd3d-48d2-b036-75b1c63f4803 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.569959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5a955-cd3d-48d2-b036-75b1c63f4803 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.570300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5a955-cd3d-48d2-b036-75b1c63f4803 admin admin] PATH_INFO: `/v3/projects/4996d609438740f79b07cec536b9681b/users/91949ba2b46140bab0bbbc465cc26b37/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.574352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5a955-cd3d-48d2-b036-75b1c63f4803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4996d609438740f79b07cec536b9681b, user_id=91949ba2b46140bab0bbbc465cc26b37, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.577340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5a955-cd3d-48d2-b036-75b1c63f4803 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.597185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1357/2684] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:22 2022] PUT /identity/v3/projects/4996d609438740f79b07cec536b9681b/users/91949ba2b46140bab0bbbc465cc26b37/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:22.608922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b608c6-f074-45ee-b3f1-0d7246fca89a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.609301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b608c6-f074-45ee-b3f1-0d7246fca89a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.609640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b608c6-f074-45ee-b3f1-0d7246fca89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.645777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-23b608c6-f074-45ee-b3f1-0d7246fca89a None None] MFA Rules not processed for user `91949ba2b46140bab0bbbc465cc26b37`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:22.645777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-23b608c6-f074-45ee-b3f1-0d7246fca89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:22.675427 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1328/2685] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:22 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:37:22.676703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088e021-484e-44ee-b219-45913996ec5e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.690173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088e021-484e-44ee-b219-45913996ec5e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.760305 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1358/2686] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:22.768084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b608c6-f074-45ee-b3f1-0d7246fca89a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.771260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-23b608c6-f074-45ee-b3f1-0d7246fca89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:22.781818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d69087-1f74-4f3d-811e-371c1bb1a314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.782588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d69087-1f74-4f3d-811e-371c1bb1a314 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.782896 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d69087-1f74-4f3d-811e-371c1bb1a314 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.783256 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d69087-1f74-4f3d-811e-371c1bb1a314 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.784522 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d69087-1f74-4f3d-811e-371c1bb1a314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.786267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d69087-1f74-4f3d-811e-371c1bb1a314 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.799012 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1329/2687] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:22.803627 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d69087-1f74-4f3d-811e-371c1bb1a314 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.815760 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d69087-1f74-4f3d-811e-371c1bb1a314 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.822795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196abb2-c391-4d80-a0f4-668e1f03268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.823571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196abb2-c391-4d80-a0f4-668e1f03268d admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.823848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196abb2-c391-4d80-a0f4-668e1f03268d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.824963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196abb2-c391-4d80-a0f4-668e1f03268d admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.825618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196abb2-c391-4d80-a0f4-668e1f03268d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.828128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196abb2-c391-4d80-a0f4-668e1f03268d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.863250 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1330/2688] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:22.863450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b608c6-f074-45ee-b3f1-0d7246fca89a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.865972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196abb2-c391-4d80-a0f4-668e1f03268d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:22.867640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf61860-8f58-4734-a407-aa140ec48e0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.868392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf61860-8f58-4734-a407-aa140ec48e0b service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.868786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf61860-8f58-4734-a407-aa140ec48e0b service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.869118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf61860-8f58-4734-a407-aa140ec48e0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.881846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf61860-8f58-4734-a407-aa140ec48e0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.883982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf61860-8f58-4734-a407-aa140ec48e0b service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:22.891089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196abb2-c391-4d80-a0f4-668e1f03268d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:22.896913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1359/2689] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:37:22.906592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374853e-fb89-41e3-98a6-ccea4416d77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:22.906592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374853e-fb89-41e3-98a6-ccea4416d77f admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:22.906592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374853e-fb89-41e3-98a6-ccea4416d77f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:22.906592 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374853e-fb89-41e3-98a6-ccea4416d77f admin admin] PATH_INFO: `/v3/projects/f8b25b8f45c84feb869633ea4ee98521/users/640d1ecc20f54d81945ba6003787ebc7/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:22.922328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374853e-fb89-41e3-98a6-ccea4416d77f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b25b8f45c84feb869633ea4ee98521, user_id=640d1ecc20f54d81945ba6003787ebc7, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:22.926127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374853e-fb89-41e3-98a6-ccea4416d77f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:23.017415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1331/2690] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:22 2022] PUT /identity/v3/projects/f8b25b8f45c84feb869633ea4ee98521/users/640d1ecc20f54d81945ba6003787ebc7/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:23.023284 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4fd10-4708-4323-90e6-7763374f4e5f None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:23.023904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4fd10-4708-4323-90e6-7763374f4e5f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:23.028567 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4fd10-4708-4323-90e6-7763374f4e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:23.083782 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-07c4fd10-4708-4323-90e6-7763374f4e5f None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:23.086871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-07c4fd10-4708-4323-90e6-7763374f4e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:23.163395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:23.193390 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374853e-fb89-41e3-98a6-ccea4416d77f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:23.197723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1360/2691] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:23.213974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4fd10-4708-4323-90e6-7763374f4e5f None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:23.215603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-07c4fd10-4708-4323-90e6-7763374f4e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:23.222780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:23.279755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374853e-fb89-41e3-98a6-ccea4416d77f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:23.289349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5115fd-680f-480a-9127-0513572e05ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:23.290008 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5115fd-680f-480a-9127-0513572e05ad service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:23.291712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5115fd-680f-480a-9127-0513572e05ad service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:23.297934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5115fd-680f-480a-9127-0513572e05ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:23.298091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4fd10-4708-4323-90e6-7763374f4e5f None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:23.298200 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9f5115fd-680f-480a-9127-0513572e05ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:23.298603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de287248-79e9-4e99-ba9f-80a5e74fe294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:23.301010 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de287248-79e9-4e99-ba9f-80a5e74fe294 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:23.301536 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de287248-79e9-4e99-ba9f-80a5e74fe294 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:23.302326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de287248-79e9-4e99-ba9f-80a5e74fe294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:23.314805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de287248-79e9-4e99-ba9f-80a5e74fe294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:23.317833 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de287248-79e9-4e99-ba9f-80a5e74fe294 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:23.330441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1361/2692] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:23.371638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5115fd-680f-480a-9127-0513572e05ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:23.373613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5115fd-680f-480a-9127-0513572e05ad service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:23.385081 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1332/2693] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:37:24.985675 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:25.185366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:26.179129 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:27.052851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:28.128152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:29.302796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:29.667731 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebcf37-979a-48f1-88a4-87305a0221f0 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:29.668011 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebcf37-979a-48f1-88a4-87305a0221f0 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:29.668281 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebcf37-979a-48f1-88a4-87305a0221f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:29.709472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-72ebcf37-979a-48f1-88a4-87305a0221f0 None None] MFA Rules not processed for user `640d1ecc20f54d81945ba6003787ebc7`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:29.710743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-72ebcf37-979a-48f1-88a4-87305a0221f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:29.816632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1362/2694] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:29.821290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5115fd-680f-480a-9127-0513572e05ad service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:29.833173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5115fd-680f-480a-9127-0513572e05ad service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:29.838206 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6c74c-ad60-4472-a201-2eae40f68b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:29.838904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6c74c-ad60-4472-a201-2eae40f68b10 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:29.839249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6c74c-ad60-4472-a201-2eae40f68b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:29.839621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6c74c-ad60-4472-a201-2eae40f68b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:29.840309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6c74c-ad60-4472-a201-2eae40f68b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:29.842457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6c74c-ad60-4472-a201-2eae40f68b10 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:29.921163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1333/2695] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:37:29 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:29.922383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebcf37-979a-48f1-88a4-87305a0221f0 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:29.935602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebcf37-979a-48f1-88a4-87305a0221f0 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:29.995605 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7889050-9f3a-4b37-97ad-5fb0dd624f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:29.996221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7889050-9f3a-4b37-97ad-5fb0dd624f39 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:29.999531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7889050-9f3a-4b37-97ad-5fb0dd624f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:29.999531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7889050-9f3a-4b37-97ad-5fb0dd624f39 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:29.999531 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7889050-9f3a-4b37-97ad-5fb0dd624f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:29.999978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7889050-9f3a-4b37-97ad-5fb0dd624f39 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:30.076528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1363/2696] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:37:29 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:37:30.081115 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6c74c-ad60-4472-a201-2eae40f68b10 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:30.092136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6c74c-ad60-4472-a201-2eae40f68b10 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:30.097484 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ebf6d-fd80-421d-a116-fe22e28a81f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:30.098277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ebf6d-fd80-421d-a116-fe22e28a81f1 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:30.098848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ebf6d-fd80-421d-a116-fe22e28a81f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:30.099078 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ebf6d-fd80-421d-a116-fe22e28a81f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:30.099724 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ebf6d-fd80-421d-a116-fe22e28a81f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:30.101318 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ebf6d-fd80-421d-a116-fe22e28a81f1 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:30.112937 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1334/2697] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:37:30.117185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7889050-9f3a-4b37-97ad-5fb0dd624f39 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:30.128373 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7889050-9f3a-4b37-97ad-5fb0dd624f39 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:30.133076 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fc159-4a3d-45f3-bc13-c4ddd1d9b0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:30.133732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fc159-4a3d-45f3-bc13-c4ddd1d9b0b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:30.134046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fc159-4a3d-45f3-bc13-c4ddd1d9b0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:30.134316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fc159-4a3d-45f3-bc13-c4ddd1d9b0b1 admin admin] PATH_INFO: `/v3/projects/29bafbef2b8640d7b626ff304f52af86/users/0bbea4fe1fb04d099269f7e93833765e/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:30.150330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fc159-4a3d-45f3-bc13-c4ddd1d9b0b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29bafbef2b8640d7b626ff304f52af86, user_id=0bbea4fe1fb04d099269f7e93833765e, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:30.152979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fc159-4a3d-45f3-bc13-c4ddd1d9b0b1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:30.260612 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1364/2698] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:30 2022] PUT /identity/v3/projects/29bafbef2b8640d7b626ff304f52af86/users/0bbea4fe1fb04d099269f7e93833765e/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:30.274051 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ebf6d-fd80-421d-a116-fe22e28a81f1 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:30.277977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ebf6d-fd80-421d-a116-fe22e28a81f1 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:30.308137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:30.334396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381d6bc-cbf4-43f6-ac7e-9881c61ae180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:30.335025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381d6bc-cbf4-43f6-ac7e-9881c61ae180 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:30.335269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381d6bc-cbf4-43f6-ac7e-9881c61ae180 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:30.335532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381d6bc-cbf4-43f6-ac7e-9881c61ae180 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:30.336159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381d6bc-cbf4-43f6-ac7e-9881c61ae180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:30.337826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381d6bc-cbf4-43f6-ac7e-9881c61ae180 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:30.347343 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1335/2699] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:37:30.351882 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fc159-4a3d-45f3-bc13-c4ddd1d9b0b1 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:30.362613 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fc159-4a3d-45f3-bc13-c4ddd1d9b0b1 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:30.367295 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895e515-7774-405b-894e-96372d0f77e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:30.367944 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895e515-7774-405b-894e-96372d0f77e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:30.368224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895e515-7774-405b-894e-96372d0f77e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:30.368511 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895e515-7774-405b-894e-96372d0f77e4 admin admin] PATH_INFO: `/v3/projects/29bafbef2b8640d7b626ff304f52af86/users/0bbea4fe1fb04d099269f7e93833765e/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:30.370489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895e515-7774-405b-894e-96372d0f77e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29bafbef2b8640d7b626ff304f52af86, user_id=0bbea4fe1fb04d099269f7e93833765e, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:30.373122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895e515-7774-405b-894e-96372d0f77e4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:30.383920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1365/2700] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:37:30 2022] PUT /identity/v3/projects/29bafbef2b8640d7b626ff304f52af86/users/0bbea4fe1fb04d099269f7e93833765e/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:37:35.422833 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381d6bc-cbf4-43f6-ac7e-9881c61ae180 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:35.426053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5381d6bc-cbf4-43f6-ac7e-9881c61ae180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:35.526262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381d6bc-cbf4-43f6-ac7e-9881c61ae180 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:35.630423 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:35.632619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:35.632619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:35.632619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:35.634541 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:35.658415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:35.660280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:35.672984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1336/2701] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:37.225871 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6f35c-5573-423a-8d77-85b0051ad2cc None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:37.227945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6f35c-5573-423a-8d77-85b0051ad2cc None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:37.228430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6f35c-5573-423a-8d77-85b0051ad2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:37.272316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4da6f35c-5573-423a-8d77-85b0051ad2cc None None] MFA Rules not processed for user `0bbea4fe1fb04d099269f7e93833765e`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:37.273560 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4da6f35c-5573-423a-8d77-85b0051ad2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:37.311691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:37.402231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1366/2702] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:37.411437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:37.411986 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:37.483945 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:37.513429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe7704-c107-4888-969d-d9fa582019df service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:37.517779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b854485-677c-4ae0-8924-2069e7de160e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73bfe655c6d34918b21acb8748b9156d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:37.518970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b854485-677c-4ae0-8924-2069e7de160e service glance] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:37.519248 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b854485-677c-4ae0-8924-2069e7de160e service glance] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:37.519577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b854485-677c-4ae0-8924-2069e7de160e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:37.585153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b854485-677c-4ae0-8924-2069e7de160e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:37.586805 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b854485-677c-4ae0-8924-2069e7de160e service glance] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:37.598212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1337/2703] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:37.894910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483de5c-2a82-4ead-9759-60eaedfda93a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:37.895439 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483de5c-2a82-4ead-9759-60eaedfda93a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:37.898627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483de5c-2a82-4ead-9759-60eaedfda93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:37.952738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c483de5c-2a82-4ead-9759-60eaedfda93a None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:37.954007 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c483de5c-2a82-4ead-9759-60eaedfda93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:38.026829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1367/2704] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:37:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:38.232821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b854485-677c-4ae0-8924-2069e7de160e service glance] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:38.234349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6b854485-677c-4ae0-8924-2069e7de160e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:38.329460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b854485-677c-4ae0-8924-2069e7de160e service glance] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:38.334736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e646d79-ee0b-4b56-8a5c-ea342cd84f56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:38.335345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e646d79-ee0b-4b56-8a5c-ea342cd84f56 service swift] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:38.335620 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e646d79-ee0b-4b56-8a5c-ea342cd84f56 service swift] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:38.335887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e646d79-ee0b-4b56-8a5c-ea342cd84f56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:38.344496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e646d79-ee0b-4b56-8a5c-ea342cd84f56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:38.346137 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e646d79-ee0b-4b56-8a5c-ea342cd84f56 service swift] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:38.355649 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1338/2705] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:37:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:37:47.480343 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c0f20-cffa-4e7b-a6b2-7336750edcf9 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:47.480979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c0f20-cffa-4e7b-a6b2-7336750edcf9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:47.480979 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c0f20-cffa-4e7b-a6b2-7336750edcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:47.501495 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-110c0f20-cffa-4e7b-a6b2-7336750edcf9 None None] MFA Rules not processed for user `57662358169d4830b2ebb7d594efb701`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:47.503604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-110c0f20-cffa-4e7b-a6b2-7336750edcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:47.574158 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1368/2706] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:37:47.909578 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e646d79-ee0b-4b56-8a5c-ea342cd84f56 service swift] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:47.965753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e646d79-ee0b-4b56-8a5c-ea342cd84f56 service swift] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:47.970107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8af63c-9a95-451c-9c69-730f36b33752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:47.970840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8af63c-9a95-451c-9c69-730f36b33752 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:47.971197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8af63c-9a95-451c-9c69-730f36b33752 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:47.971460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8af63c-9a95-451c-9c69-730f36b33752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:47.973264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9a8af63c-9a95-451c-9c69-730f36b33752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:47.984410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8af63c-9a95-451c-9c69-730f36b33752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:47.985855 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8af63c-9a95-451c-9c69-730f36b33752 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:47.991696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:48.007317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1339/2707] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:48.929639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c0f20-cffa-4e7b-a6b2-7336750edcf9 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:48.931164 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-110c0f20-cffa-4e7b-a6b2-7336750edcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:48.970588 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:37:49.006712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c0f20-cffa-4e7b-a6b2-7336750edcf9 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:49.011819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e823a3e-0de6-496d-90e4-637652cf10a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:49.014996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e823a3e-0de6-496d-90e4-637652cf10a1 service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:49.014996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e823a3e-0de6-496d-90e4-637652cf10a1 service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:49.014996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e823a3e-0de6-496d-90e4-637652cf10a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:49.014996 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2e823a3e-0de6-496d-90e4-637652cf10a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:49.027013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e823a3e-0de6-496d-90e4-637652cf10a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:49.028573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e823a3e-0de6-496d-90e4-637652cf10a1 service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:49.043322 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1369/2708] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:37:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:37:49.380734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fe3c8-e9f4-4332-b323-faebfd2f3e12 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:49.381255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fe3c8-e9f4-4332-b323-faebfd2f3e12 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:49.381255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fe3c8-e9f4-4332-b323-faebfd2f3e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:49.400311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-3d2fe3c8-e9f4-4332-b323-faebfd2f3e12 None None] MFA Rules not processed for user `640d1ecc20f54d81945ba6003787ebc7`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:49.401612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3d2fe3c8-e9f4-4332-b323-faebfd2f3e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:49.408881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1340/2709] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:37:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:53.246824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e59bfe-9931-4ab0-bae4-42bbdbbc55c8 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:53.247383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e59bfe-9931-4ab0-bae4-42bbdbbc55c8 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:53.247516 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e59bfe-9931-4ab0-bae4-42bbdbbc55c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:53.279812 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e8e59bfe-9931-4ab0-bae4-42bbdbbc55c8 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:37:53.281031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e8e59bfe-9931-4ab0-bae4-42bbdbbc55c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:37:53.291091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1370/2710] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:37:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:37:53.315580 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fe3c8-e9f4-4332-b323-faebfd2f3e12 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:37:53.331246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fe3c8-e9f4-4332-b323-faebfd2f3e12 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:37:53.337807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4dbb-0722-4483-a605-1dd79955a0a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:37:53.338552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4dbb-0722-4483-a605-1dd79955a0a5 service swift] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:37:53.339011 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4dbb-0722-4483-a605-1dd79955a0a5 service swift] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:37:53.339380 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4dbb-0722-4483-a605-1dd79955a0a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:37:53.351072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a4dbb-0722-4483-a605-1dd79955a0a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:37:53.353121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a4dbb-0722-4483-a605-1dd79955a0a5 service swift] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:37:53.378860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1341/2711] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:37:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:38:03.272889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=20 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:03.362468 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=9 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:10.114115 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e59bfe-9931-4ab0-bae4-42bbdbbc55c8 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:10.127414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e59bfe-9931-4ab0-bae4-42bbdbbc55c8 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:10.132961 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024217-d822-4fd2-80ba-0c80bb433e97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:10.133504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024217-d822-4fd2-80ba-0c80bb433e97 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:10.133804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024217-d822-4fd2-80ba-0c80bb433e97 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:10.134078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024217-d822-4fd2-80ba-0c80bb433e97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:10.135818 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-cc024217-d822-4fd2-80ba-0c80bb433e97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:10.147792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc024217-d822-4fd2-80ba-0c80bb433e97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:10.149392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc024217-d822-4fd2-80ba-0c80bb433e97 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:10.159702 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1371/2712] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:38:19.676698 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4dbb-0722-4483-a605-1dd79955a0a5 service swift] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:19.678152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-be2a4dbb-0722-4483-a605-1dd79955a0a5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:19.690848 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4dbb-0722-4483-a605-1dd79955a0a5 service swift] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:19.694331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1268270-0f72-41b6-a452-c2819804cdeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:19.695410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1268270-0f72-41b6-a452-c2819804cdeb service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:19.695680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1268270-0f72-41b6-a452-c2819804cdeb service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:19.695941 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1268270-0f72-41b6-a452-c2819804cdeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:19.697758 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c1268270-0f72-41b6-a452-c2819804cdeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:19.766147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1268270-0f72-41b6-a452-c2819804cdeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:19.767618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1268270-0f72-41b6-a452-c2819804cdeb service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:19.778800 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1342/2713] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:38:20.155307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=6 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:21.365055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024217-d822-4fd2-80ba-0c80bb433e97 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:21.430353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024217-d822-4fd2-80ba-0c80bb433e97 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:21.434970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:21.435597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:21.435870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:21.436137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] PATH_INFO: `/v3/users/8c37ba4362ea49fa9eacbfb2e98dc581` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:21.437249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c37ba4362ea49fa9eacbfb2e98dc581)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:21.439205 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:21.638231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c37ba4362ea49fa9eacbfb2e98dc581'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:21.698138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c37ba4362ea49fa9eacbfb2e98dc581'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:21.705985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c37ba4362ea49fa9eacbfb2e98dc581'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:21.718716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c37ba4362ea49fa9eacbfb2e98dc581'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:21.719430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c37ba4362ea49fa9eacbfb2e98dc581'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:21.727171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1372/2714] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:38:21 2022] DELETE /identity/v3/users/8c37ba4362ea49fa9eacbfb2e98dc581 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:21.731691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e76ab-d43d-430c-b394-4e1e6fe0a31b None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:21.732133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e76ab-d43d-430c-b394-4e1e6fe0a31b None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:21.732239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e76ab-d43d-430c-b394-4e1e6fe0a31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:21.751320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-2a0e76ab-d43d-430c-b394-4e1e6fe0a31b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:21.752830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2a0e76ab-d43d-430c-b394-4e1e6fe0a31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:21.822862 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1343/2715] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:38:21.832549 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:21.833991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:21.897185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852948dd-9c27-443a-8f5d-aa16fea1af8d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:21.901073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15edb5-350a-4a29-a11c-53c011b83821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:21.901749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15edb5-350a-4a29-a11c-53c011b83821 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:21.902280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15edb5-350a-4a29-a11c-53c011b83821 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:21.902489 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15edb5-350a-4a29-a11c-53c011b83821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:21.914940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee15edb5-350a-4a29-a11c-53c011b83821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:21.916469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee15edb5-350a-4a29-a11c-53c011b83821 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:21.927786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1373/2716] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:38:22.479673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e76ab-d43d-430c-b394-4e1e6fe0a31b None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:22.481440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2a0e76ab-d43d-430c-b394-4e1e6fe0a31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:22.499682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e76ab-d43d-430c-b394-4e1e6fe0a31b None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:22.510369 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:22.511083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:22.511434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:22.511771 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] PATH_INFO: `/v3/projects/dbe2f85ec90a49f4940d8d3067feb7cb` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:22.513309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe2f85ec90a49f4940d8d3067feb7cb)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:22.515549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:22.615708 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe2f85ec90a49f4940d8d3067feb7cb', 'request_id': 'req-36bcf276-0f91-4101-8f6c-4773ce00079a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:22.623309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe2f85ec90a49f4940d8d3067feb7cb', 'request_id': 'req-36bcf276-0f91-4101-8f6c-4773ce00079a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:22.698464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe2f85ec90a49f4940d8d3067feb7cb', 'request_id': 'req-36bcf276-0f91-4101-8f6c-4773ce00079a'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:22.716310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] The token cache is being invalidate because project dbe2f85ec90a49f4940d8d3067feb7cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:38:22.722984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:22.722984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:22.722984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1344/2717] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:38:22 2022] DELETE /identity/v3/projects/dbe2f85ec90a49f4940d8d3067feb7cb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:22.724933 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15edb5-350a-4a29-a11c-53c011b83821 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:22.738660 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee15edb5-350a-4a29-a11c-53c011b83821 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:22.758152 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:22.797224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:22.797879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:22.798163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:22.798432 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] PATH_INFO: `/v3/users/91949ba2b46140bab0bbbc465cc26b37` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:22.799607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=91949ba2b46140bab0bbbc465cc26b37)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:22.801539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:22.940853 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91949ba2b46140bab0bbbc465cc26b37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.035117 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91949ba2b46140bab0bbbc465cc26b37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.042976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91949ba2b46140bab0bbbc465cc26b37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.056330 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91949ba2b46140bab0bbbc465cc26b37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.058000 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95e317f4-2527-4d96-8909-4520eb25a7ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91949ba2b46140bab0bbbc465cc26b37'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.064134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1374/2718] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:38:22 2022] DELETE /identity/v3/users/91949ba2b46140bab0bbbc465cc26b37 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:38:23.528262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:23.529816 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:23.544963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcf276-0f91-4101-8f6c-4773ce00079a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:23.562122 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:23.611864 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:23.612737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:23.613015 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:23.613282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] PATH_INFO: `/v3/projects/4996d609438740f79b07cec536b9681b` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:23.614949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4996d609438740f79b07cec536b9681b)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:23.617118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:23.738445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4996d609438740f79b07cec536b9681b', 'request_id': 'req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.744755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4996d609438740f79b07cec536b9681b', 'request_id': 'req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.805761 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4996d609438740f79b07cec536b9681b', 'request_id': 'req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.816481 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] The token cache is being invalidate because project 4996d609438740f79b07cec536b9681b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:38:23.816783 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.817397 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d3a6939c-6c58-4a6b-964c-4b96ccb8a5b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:23.818892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1345/2719] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:38:23 2022] DELETE /identity/v3/projects/4996d609438740f79b07cec536b9681b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:23.823646 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a826a5-c58a-421b-9a46-749e2e5582ea None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:23.824134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a826a5-c58a-421b-9a46-749e2e5582ea None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:23.824215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a826a5-c58a-421b-9a46-749e2e5582ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:23.844368 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c3a826a5-c58a-421b-9a46-749e2e5582ea None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:23.845581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c3a826a5-c58a-421b-9a46-749e2e5582ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:23.904639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1375/2720] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:38:23.914231 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac20c46-858f-47d3-ae30-bb5acdca9384 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:23.914516 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac20c46-858f-47d3-ae30-bb5acdca9384 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:23.914789 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac20c46-858f-47d3-ae30-bb5acdca9384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:23.934091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-4ac20c46-858f-47d3-ae30-bb5acdca9384 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:23.935300 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4ac20c46-858f-47d3-ae30-bb5acdca9384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:23.943435 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1346/2721] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:38:23.947521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a826a5-c58a-421b-9a46-749e2e5582ea None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:23.958175 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a826a5-c58a-421b-9a46-749e2e5582ea None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:23.963198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d8df6-2f7e-4ccd-b7bb-de276b1b7058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:23.963781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d8df6-2f7e-4ccd-b7bb-de276b1b7058 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:23.964053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d8df6-2f7e-4ccd-b7bb-de276b1b7058 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:23.964340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d8df6-2f7e-4ccd-b7bb-de276b1b7058 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:23.964974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d8df6-2f7e-4ccd-b7bb-de276b1b7058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:38:23.965273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d8df6-2f7e-4ccd-b7bb-de276b1b7058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:23.966877 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d8df6-2f7e-4ccd-b7bb-de276b1b7058 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:23.978909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1376/2722] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:38:23.983128 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac20c46-858f-47d3-ae30-bb5acdca9384 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:23.992083 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac20c46-858f-47d3-ae30-bb5acdca9384 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:23.996663 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1ca1-d560-46a9-af4c-1794cb5db561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:23.997269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1ca1-d560-46a9-af4c-1794cb5db561 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:23.997539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1ca1-d560-46a9-af4c-1794cb5db561 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:23.997819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1ca1-d560-46a9-af4c-1794cb5db561 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:23.998428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a1ca1-d560-46a9-af4c-1794cb5db561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:24.000293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a1ca1-d560-46a9-af4c-1794cb5db561 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:24.094515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1347/2723] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:38:23 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:38:24.098924 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d8df6-2f7e-4ccd-b7bb-de276b1b7058 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:24.109014 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d8df6-2f7e-4ccd-b7bb-de276b1b7058 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:24.134738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:24.163788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc0a97-695f-48f1-af90-8b2b13805f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:24.164387 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc0a97-695f-48f1-af90-8b2b13805f73 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:24.164680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc0a97-695f-48f1-af90-8b2b13805f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:24.165058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc0a97-695f-48f1-af90-8b2b13805f73 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:24.165677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc0a97-695f-48f1-af90-8b2b13805f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:24.167594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc0a97-695f-48f1-af90-8b2b13805f73 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:24.287498 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1377/2724] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:38:24 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:38:24.291283 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1ca1-d560-46a9-af4c-1794cb5db561 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:24.301611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1ca1-d560-46a9-af4c-1794cb5db561 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:24.306322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfff61-c6b1-41b0-9e48-de455a51a10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:24.306916 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfff61-c6b1-41b0-9e48-de455a51a10d admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:24.307196 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfff61-c6b1-41b0-9e48-de455a51a10d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:24.307463 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfff61-c6b1-41b0-9e48-de455a51a10d admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:24.308102 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfff61-c6b1-41b0-9e48-de455a51a10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:24.313929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfff61-c6b1-41b0-9e48-de455a51a10d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:24.327930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1348/2725] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:38:24.329198 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc0a97-695f-48f1-af90-8b2b13805f73 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:24.339845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc0a97-695f-48f1-af90-8b2b13805f73 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:24.344518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5169bd-f902-43a7-8d52-49c04954e0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:24.354307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5169bd-f902-43a7-8d52-49c04954e0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:24.354307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5169bd-f902-43a7-8d52-49c04954e0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:24.354307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5169bd-f902-43a7-8d52-49c04954e0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:24.354307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5169bd-f902-43a7-8d52-49c04954e0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:24.354307 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5169bd-f902-43a7-8d52-49c04954e0f5 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:24.358746 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1378/2726] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:38:24.363034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfff61-c6b1-41b0-9e48-de455a51a10d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:24.373171 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfff61-c6b1-41b0-9e48-de455a51a10d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:24.377966 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fe3e8-b542-4c53-8a31-edcd8aa3ad4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:24.378609 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fe3e8-b542-4c53-8a31-edcd8aa3ad4d admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:24.378877 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fe3e8-b542-4c53-8a31-edcd8aa3ad4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:24.379143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fe3e8-b542-4c53-8a31-edcd8aa3ad4d admin admin] PATH_INFO: `/v3/projects/9d3539da989849d28b2b728e257d010d/users/ba0f1dd6bd324a3c9f3babbf23d96d79/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:24.395288 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fe3e8-b542-4c53-8a31-edcd8aa3ad4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3539da989849d28b2b728e257d010d, user_id=ba0f1dd6bd324a3c9f3babbf23d96d79, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:24.398947 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fe3e8-b542-4c53-8a31-edcd8aa3ad4d admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:24.493639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1349/2727] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:38:24 2022] PUT /identity/v3/projects/9d3539da989849d28b2b728e257d010d/users/ba0f1dd6bd324a3c9f3babbf23d96d79/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:24.504163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c133c-5e15-4072-882f-2078df0c74a3 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:24.504503 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c133c-5e15-4072-882f-2078df0c74a3 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:24.504774 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c133c-5e15-4072-882f-2078df0c74a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:24.532362 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-0b2c133c-5e15-4072-882f-2078df0c74a3 None None] MFA Rules not processed for user `ba0f1dd6bd324a3c9f3babbf23d96d79`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:24.533552 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0b2c133c-5e15-4072-882f-2078df0c74a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:24.604666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1379/2728] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:38:24.609164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fe3e8-b542-4c53-8a31-edcd8aa3ad4d admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:24.654753 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:24.671155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fe3e8-b542-4c53-8a31-edcd8aa3ad4d admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:24.676033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0da332-e8b4-4fac-bc94-c65d954eb984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:24.676630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0da332-e8b4-4fac-bc94-c65d954eb984 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:24.676908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0da332-e8b4-4fac-bc94-c65d954eb984 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:24.677179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0da332-e8b4-4fac-bc94-c65d954eb984 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:24.677844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0da332-e8b4-4fac-bc94-c65d954eb984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:24.679754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0da332-e8b4-4fac-bc94-c65d954eb984 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:24.773824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1350/2729] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:38:24 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:38:24.777286 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c133c-5e15-4072-882f-2078df0c74a3 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:24.787847 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c133c-5e15-4072-882f-2078df0c74a3 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:24.844834 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652a983-1d1f-4e70-9a98-5ca5c97e2e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:24.845475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652a983-1d1f-4e70-9a98-5ca5c97e2e29 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:24.845632 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652a983-1d1f-4e70-9a98-5ca5c97e2e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:24.846005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652a983-1d1f-4e70-9a98-5ca5c97e2e29 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:24.846623 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652a983-1d1f-4e70-9a98-5ca5c97e2e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:24.848522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652a983-1d1f-4e70-9a98-5ca5c97e2e29 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:24.932273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1380/2730] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:38:24 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:38:24.936415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0da332-e8b4-4fac-bc94-c65d954eb984 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:24.947506 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0da332-e8b4-4fac-bc94-c65d954eb984 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:24.952177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55707f0f-b93d-4888-b591-2ee0a765f47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:24.952766 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55707f0f-b93d-4888-b591-2ee0a765f47a admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:24.953073 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55707f0f-b93d-4888-b591-2ee0a765f47a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:24.953345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55707f0f-b93d-4888-b591-2ee0a765f47a admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:24.954050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55707f0f-b93d-4888-b591-2ee0a765f47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:24.955643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55707f0f-b93d-4888-b591-2ee0a765f47a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:24.966374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1351/2731] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:38:24.970611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652a983-1d1f-4e70-9a98-5ca5c97e2e29 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:24.981429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652a983-1d1f-4e70-9a98-5ca5c97e2e29 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:24.986402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c289-1f0a-44e2-8789-62d04bb69f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:24.987058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c289-1f0a-44e2-8789-62d04bb69f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:24.987447 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c289-1f0a-44e2-8789-62d04bb69f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:24.987725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c289-1f0a-44e2-8789-62d04bb69f8a admin admin] PATH_INFO: `/v3/projects/91f75e1b476644029c1bcdbdfd6dce55/users/58df35be93a14ae196d7bb3ebc1b37f6/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:25.002499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9c289-1f0a-44e2-8789-62d04bb69f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f75e1b476644029c1bcdbdfd6dce55, user_id=58df35be93a14ae196d7bb3ebc1b37f6, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:25.005280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9c289-1f0a-44e2-8789-62d04bb69f8a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:25.090901 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1381/2732] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:38:24 2022] PUT /identity/v3/projects/91f75e1b476644029c1bcdbdfd6dce55/users/58df35be93a14ae196d7bb3ebc1b37f6/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:25.095138 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55707f0f-b93d-4888-b591-2ee0a765f47a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:25.105455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55707f0f-b93d-4888-b591-2ee0a765f47a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:25.166430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd384181-4817-41fe-8be9-ede1b84c9ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:25.167038 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd384181-4817-41fe-8be9-ede1b84c9ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:25.167317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd384181-4817-41fe-8be9-ede1b84c9ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:25.167589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd384181-4817-41fe-8be9-ede1b84c9ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:25.168260 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd384181-4817-41fe-8be9-ede1b84c9ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:25.169901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd384181-4817-41fe-8be9-ede1b84c9ce6 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:25.180735 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1352/2733] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:38:25.184657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c289-1f0a-44e2-8789-62d04bb69f8a admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:25.190035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:25.195145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9c289-1f0a-44e2-8789-62d04bb69f8a admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:25.200312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e272b-fdce-407b-9a9f-10d5c2f8af40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:25.200945 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e272b-fdce-407b-9a9f-10d5c2f8af40 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:25.201236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e272b-fdce-407b-9a9f-10d5c2f8af40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:25.201482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e272b-fdce-407b-9a9f-10d5c2f8af40 admin admin] PATH_INFO: `/v3/projects/91f75e1b476644029c1bcdbdfd6dce55/users/58df35be93a14ae196d7bb3ebc1b37f6/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:25.203429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e272b-fdce-407b-9a9f-10d5c2f8af40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f75e1b476644029c1bcdbdfd6dce55, user_id=58df35be93a14ae196d7bb3ebc1b37f6, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:25.206073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e272b-fdce-407b-9a9f-10d5c2f8af40 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:25.215942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1382/2734] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:38:25 2022] PUT /identity/v3/projects/91f75e1b476644029c1bcdbdfd6dce55/users/58df35be93a14ae196d7bb3ebc1b37f6/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:25.227414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b3ef5-ad5f-43ae-b31e-dc52571a14a0 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:25.227703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b3ef5-ad5f-43ae-b31e-dc52571a14a0 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:25.227999 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b3ef5-ad5f-43ae-b31e-dc52571a14a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:25.257257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-ae1b3ef5-ad5f-43ae-b31e-dc52571a14a0 None None] MFA Rules not processed for user `58df35be93a14ae196d7bb3ebc1b37f6`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:25.257394 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ae1b3ef5-ad5f-43ae-b31e-dc52571a14a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:25.328259 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1353/2735] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:38:25.339750 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e272b-fdce-407b-9a9f-10d5c2f8af40 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:25.341517 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c57e272b-fdce-407b-9a9f-10d5c2f8af40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:25.398604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e272b-fdce-407b-9a9f-10d5c2f8af40 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:25.402139 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d16e8-0bd8-4e6d-8f6b-2e1fbbe2c147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:25.402729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d16e8-0bd8-4e6d-8f6b-2e1fbbe2c147 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:25.403003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d16e8-0bd8-4e6d-8f6b-2e1fbbe2c147 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:25.403266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d16e8-0bd8-4e6d-8f6b-2e1fbbe2c147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:25.412088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d16e8-0bd8-4e6d-8f6b-2e1fbbe2c147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:25.420507 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d16e8-0bd8-4e6d-8f6b-2e1fbbe2c147 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:25.422726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1383/2736] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:38:25.706889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b3ef5-ad5f-43ae-b31e-dc52571a14a0 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:25.716784 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b3ef5-ad5f-43ae-b31e-dc52571a14a0 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:25.720359 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228dd13-ab57-4fdf-ac19-0f7446ffb1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:25.720950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228dd13-ab57-4fdf-ac19-0f7446ffb1ba service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:25.721224 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228dd13-ab57-4fdf-ac19-0f7446ffb1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:25.721493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228dd13-ab57-4fdf-ac19-0f7446ffb1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:25.761973 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:25.780859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228dd13-ab57-4fdf-ac19-0f7446ffb1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:25.782334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228dd13-ab57-4fdf-ac19-0f7446ffb1ba service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:25.791884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1354/2737] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:38:29.232243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d16e8-0bd8-4e6d-8f6b-2e1fbbe2c147 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:29.294028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d16e8-0bd8-4e6d-8f6b-2e1fbbe2c147 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:29.298634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:29.299255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:29.299537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:29.299842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] PATH_INFO: `/v3/users/ba0f1dd6bd324a3c9f3babbf23d96d79` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:29.300992 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0f1dd6bd324a3c9f3babbf23d96d79)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:29.303041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:29.495946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0f1dd6bd324a3c9f3babbf23d96d79'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:29.577983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0f1dd6bd324a3c9f3babbf23d96d79'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:29.589968 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0f1dd6bd324a3c9f3babbf23d96d79'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:29.603258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0f1dd6bd324a3c9f3babbf23d96d79'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:29.603993 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0f1dd6bd324a3c9f3babbf23d96d79'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:29.611424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1384/2738] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:38:29 2022] DELETE /identity/v3/users/ba0f1dd6bd324a3c9f3babbf23d96d79 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:29.637963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06d2a0-bcf4-429f-9da4-56284c7956b4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:29.638808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06d2a0-bcf4-429f-9da4-56284c7956b4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:29.639084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06d2a0-bcf4-429f-9da4-56284c7956b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:29.659118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-cc06d2a0-bcf4-429f-9da4-56284c7956b4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:29.660335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-cc06d2a0-bcf4-429f-9da4-56284c7956b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:29.677316 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:29.678780 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:29.692267 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:29.723543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1355/2739] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:38:29.733980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06d2a0-bcf4-429f-9da4-56284c7956b4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:29.741583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d72a-cf0e-4122-9d45-adf1d21e095d admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:29.743886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06d2a0-bcf4-429f-9da4-56284c7956b4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:29.745761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d535d-657a-4f69-a4d9-c4911a02106a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:29.746475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d535d-657a-4f69-a4d9-c4911a02106a service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:29.746786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d535d-657a-4f69-a4d9-c4911a02106a service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:29.747096 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d535d-657a-4f69-a4d9-c4911a02106a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:29.748570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873d125-8e91-41ab-a8ad-7164f5653cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:29.749228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873d125-8e91-41ab-a8ad-7164f5653cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:29.749543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873d125-8e91-41ab-a8ad-7164f5653cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:29.749891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873d125-8e91-41ab-a8ad-7164f5653cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:29.750696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-db0d535d-657a-4f69-a4d9-c4911a02106a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:29.761907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873d125-8e91-41ab-a8ad-7164f5653cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:29.763360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873d125-8e91-41ab-a8ad-7164f5653cf6 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:29.778656 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1356/2740] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:38:29.832023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d535d-657a-4f69-a4d9-c4911a02106a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:29.833767 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d535d-657a-4f69-a4d9-c4911a02106a service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:29.847031 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1385/2741] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:38:30.876320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873d125-8e91-41ab-a8ad-7164f5653cf6 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:30.877817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-f873d125-8e91-41ab-a8ad-7164f5653cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:30.903497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873d125-8e91-41ab-a8ad-7164f5653cf6 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:30.907956 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:30.908587 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:30.908859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:30.909124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] PATH_INFO: `/v3/projects/9d3539da989849d28b2b728e257d010d` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:30.910377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3539da989849d28b2b728e257d010d)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:30.914693 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:31.090047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3539da989849d28b2b728e257d010d', 'request_id': 'req-413cca46-7974-4585-a0ec-a8a0f8f0af68'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.100167 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3539da989849d28b2b728e257d010d', 'request_id': 'req-413cca46-7974-4585-a0ec-a8a0f8f0af68'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.178336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3539da989849d28b2b728e257d010d', 'request_id': 'req-413cca46-7974-4585-a0ec-a8a0f8f0af68'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.189847 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] The token cache is being invalidate because project 9d3539da989849d28b2b728e257d010d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:38:31.190134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.190764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.192365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1357/2742] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:38:30 2022] DELETE /identity/v3/projects/9d3539da989849d28b2b728e257d010d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:31.196485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d535d-657a-4f69-a4d9-c4911a02106a service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:31.207051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d535d-657a-4f69-a4d9-c4911a02106a service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:31.277051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:31.277711 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:31.278049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:31.278324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] PATH_INFO: `/v3/users/58df35be93a14ae196d7bb3ebc1b37f6` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:31.279597 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58df35be93a14ae196d7bb3ebc1b37f6)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:31.281587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:31.494442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58df35be93a14ae196d7bb3ebc1b37f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.572496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58df35be93a14ae196d7bb3ebc1b37f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.580599 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58df35be93a14ae196d7bb3ebc1b37f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.581773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:31.595647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58df35be93a14ae196d7bb3ebc1b37f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.596392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4477ebb0-50de-4960-834c-1257187cc7f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58df35be93a14ae196d7bb3ebc1b37f6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:31.602664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1386/2743] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:38:31 2022] DELETE /identity/v3/users/58df35be93a14ae196d7bb3ebc1b37f6 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:32.060617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:32.062252 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:32.084340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cca46-7974-4585-a0ec-a8a0f8f0af68 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:32.138193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:32.151145 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:32.151993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:32.152282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:32.152551 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] PATH_INFO: `/v3/projects/91f75e1b476644029c1bcdbdfd6dce55` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:32.153972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f75e1b476644029c1bcdbdfd6dce55)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:32.155874 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:32.259320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f75e1b476644029c1bcdbdfd6dce55', 'request_id': 'req-8fcbae61-6bf7-477e-8234-c2056d9cff75'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:32.265764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f75e1b476644029c1bcdbdfd6dce55', 'request_id': 'req-8fcbae61-6bf7-477e-8234-c2056d9cff75'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:32.337208 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f75e1b476644029c1bcdbdfd6dce55', 'request_id': 'req-8fcbae61-6bf7-477e-8234-c2056d9cff75'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:32.348052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] The token cache is being invalidate because project 91f75e1b476644029c1bcdbdfd6dce55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:38:32.348335 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:32.349215 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-8fcbae61-6bf7-477e-8234-c2056d9cff75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:38:32.350936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1358/2744] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:38:32 2022] DELETE /identity/v3/projects/91f75e1b476644029c1bcdbdfd6dce55 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:32.358105 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8e12c-f237-4dc0-ade1-988c81333a3a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:32.358578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8e12c-f237-4dc0-ade1-988c81333a3a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:32.358965 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8e12c-f237-4dc0-ade1-988c81333a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:32.379551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-9de8e12c-f237-4dc0-ade1-988c81333a3a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:32.380761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-9de8e12c-f237-4dc0-ade1-988c81333a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:32.443005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1387/2745] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:38:32.450909 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65701e5c-0bd3-4262-b056-6f8680799350 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:32.451192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65701e5c-0bd3-4262-b056-6f8680799350 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:32.451674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65701e5c-0bd3-4262-b056-6f8680799350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:32.471161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-65701e5c-0bd3-4262-b056-6f8680799350 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:32.472353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-65701e5c-0bd3-4262-b056-6f8680799350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:32.481185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1359/2746] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:38:32.485419 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8e12c-f237-4dc0-ade1-988c81333a3a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:32.495985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8e12c-f237-4dc0-ade1-988c81333a3a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:32.500796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f07f81-cc60-4ccc-bf65-4147ee80e7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:32.501394 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f07f81-cc60-4ccc-bf65-4147ee80e7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:32.501703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f07f81-cc60-4ccc-bf65-4147ee80e7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:32.501998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f07f81-cc60-4ccc-bf65-4147ee80e7a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:32.502641 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f07f81-cc60-4ccc-bf65-4147ee80e7a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:38:32.502942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f07f81-cc60-4ccc-bf65-4147ee80e7a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:32.505123 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f07f81-cc60-4ccc-bf65-4147ee80e7a8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:32.517145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1388/2747] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:38:32.521568 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65701e5c-0bd3-4262-b056-6f8680799350 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:32.531013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65701e5c-0bd3-4262-b056-6f8680799350 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:32.535577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874f418-508e-4c53-8dfb-e2ffd692d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:32.536178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874f418-508e-4c53-8dfb-e2ffd692d7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:32.536447 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874f418-508e-4c53-8dfb-e2ffd692d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:32.536712 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874f418-508e-4c53-8dfb-e2ffd692d7a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:32.537322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874f418-508e-4c53-8dfb-e2ffd692d7a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:32.539263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874f418-508e-4c53-8dfb-e2ffd692d7a4 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:32.643140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1360/2748] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:38:32 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:38:32.649919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f07f81-cc60-4ccc-bf65-4147ee80e7a8 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:32.660618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f07f81-cc60-4ccc-bf65-4147ee80e7a8 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:32.728698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e0350-4132-41cf-a874-57912be93171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:32.729892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e0350-4132-41cf-a874-57912be93171 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:32.730185 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e0350-4132-41cf-a874-57912be93171 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:32.730486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e0350-4132-41cf-a874-57912be93171 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:32.731652 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e0350-4132-41cf-a874-57912be93171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:32.734213 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e0350-4132-41cf-a874-57912be93171 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:32.882451 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1389/2749] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:38:32 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:38:32.887377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874f418-508e-4c53-8dfb-e2ffd692d7a4 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:32.907160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874f418-508e-4c53-8dfb-e2ffd692d7a4 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:32.911987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc00959-5e8e-4a93-9632-7e500d46206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:32.912589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc00959-5e8e-4a93-9632-7e500d46206b admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:32.912866 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc00959-5e8e-4a93-9632-7e500d46206b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:32.913134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc00959-5e8e-4a93-9632-7e500d46206b admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:32.913882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc00959-5e8e-4a93-9632-7e500d46206b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:32.915509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc00959-5e8e-4a93-9632-7e500d46206b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:32.926117 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1361/2750] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:38:32.930375 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e0350-4132-41cf-a874-57912be93171 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:32.946430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e0350-4132-41cf-a874-57912be93171 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:32.953534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a3d10-a159-46d4-9a62-8b9021ecf0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:32.954806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a3d10-a159-46d4-9a62-8b9021ecf0fa admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:32.954806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a3d10-a159-46d4-9a62-8b9021ecf0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:32.955102 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a3d10-a159-46d4-9a62-8b9021ecf0fa admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:32.955764 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a3d10-a159-46d4-9a62-8b9021ecf0fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:32.960733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a3d10-a159-46d4-9a62-8b9021ecf0fa admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:32.973905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1390/2751] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:38:32.975836 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc00959-5e8e-4a93-9632-7e500d46206b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:32.995960 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc00959-5e8e-4a93-9632-7e500d46206b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:33.002719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5765fc-894b-40a2-b9e9-033a57b1d975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:33.003414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5765fc-894b-40a2-b9e9-033a57b1d975 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:33.003725 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5765fc-894b-40a2-b9e9-033a57b1d975 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:33.003987 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5765fc-894b-40a2-b9e9-033a57b1d975 admin admin] PATH_INFO: `/v3/projects/2a87a1912d31432eaa11f28865ca7d98/users/3be82c73106c48fa83ab08e96ecbfe81/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:33.026729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5765fc-894b-40a2-b9e9-033a57b1d975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a87a1912d31432eaa11f28865ca7d98, user_id=3be82c73106c48fa83ab08e96ecbfe81, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:33.031842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5765fc-894b-40a2-b9e9-033a57b1d975 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:33.107505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1362/2752] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:38:32 2022] PUT /identity/v3/projects/2a87a1912d31432eaa11f28865ca7d98/users/3be82c73106c48fa83ab08e96ecbfe81/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:38:33.112688 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544c6d-15eb-4925-aeae-cb698cced62e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:33.113051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544c6d-15eb-4925-aeae-cb698cced62e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:33.113336 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544c6d-15eb-4925-aeae-cb698cced62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:33.134442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-a2544c6d-15eb-4925-aeae-cb698cced62e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:33.135695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a2544c6d-15eb-4925-aeae-cb698cced62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:33.184130 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:33.206352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1391/2753] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:38:33.222417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5765fc-894b-40a2-b9e9-033a57b1d975 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:33.225027 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ff5765fc-894b-40a2-b9e9-033a57b1d975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:33.319718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5765fc-894b-40a2-b9e9-033a57b1d975 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:33.323745 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338696c-37b3-4db3-be7a-1b3344242ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:33.324414 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338696c-37b3-4db3-be7a-1b3344242ebf service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:33.324717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338696c-37b3-4db3-be7a-1b3344242ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:33.324968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338696c-37b3-4db3-be7a-1b3344242ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:33.336687 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338696c-37b3-4db3-be7a-1b3344242ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:33.338225 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338696c-37b3-4db3-be7a-1b3344242ebf service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:33.354067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1363/2754] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:38:34.841633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234126a-c990-4d2b-b42a-87992bfe97d7 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:34.842145 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234126a-c990-4d2b-b42a-87992bfe97d7 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:34.842285 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234126a-c990-4d2b-b42a-87992bfe97d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:34.872585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1234126a-c990-4d2b-b42a-87992bfe97d7 None None] MFA Rules not processed for user `3be82c73106c48fa83ab08e96ecbfe81`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:34.873827 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1234126a-c990-4d2b-b42a-87992bfe97d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:34.947647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1392/2755] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:38:34.959305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338696c-37b3-4db3-be7a-1b3344242ebf service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:34.960710 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2338696c-37b3-4db3-be7a-1b3344242ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:34.972076 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:35.020278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338696c-37b3-4db3-be7a-1b3344242ebf service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:35.023965 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81515e87-2673-48df-9f22-06581b966284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:35.024561 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515e87-2673-48df-9f22-06581b966284 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:35.024952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515e87-2673-48df-9f22-06581b966284 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:35.025221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515e87-2673-48df-9f22-06581b966284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:35.033654 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81515e87-2673-48df-9f22-06581b966284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:35.035141 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81515e87-2673-48df-9f22-06581b966284 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:35.045952 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1364/2756] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:38:36.007474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234126a-c990-4d2b-b42a-87992bfe97d7 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:38:36.008940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1234126a-c990-4d2b-b42a-87992bfe97d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:36.105708 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234126a-c990-4d2b-b42a-87992bfe97d7 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:38:36.112244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca129ac-a3b6-4ca9-b602-1b5e6d6ad502 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:38:36.114603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca129ac-a3b6-4ca9-b602-1b5e6d6ad502 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:36.114603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca129ac-a3b6-4ca9-b602-1b5e6d6ad502 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:36.114603 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca129ac-a3b6-4ca9-b602-1b5e6d6ad502 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:36.115786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-0ca129ac-a3b6-4ca9-b602-1b5e6d6ad502 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:36.131244 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca129ac-a3b6-4ca9-b602-1b5e6d6ad502 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:38:36.132982 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca129ac-a3b6-4ca9-b602-1b5e6d6ad502 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:38:36.146406 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1393/2757] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:38:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:38:37.064030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f05c3-29af-404f-95b6-58aea11e1e8a None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:37.074108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f05c3-29af-404f-95b6-58aea11e1e8a None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:37.074108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f05c3-29af-404f-95b6-58aea11e1e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:37.084298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-e29f05c3-29af-404f-95b6-58aea11e1e8a None None] MFA Rules not processed for user `3be82c73106c48fa83ab08e96ecbfe81`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:37.085571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e29f05c3-29af-404f-95b6-58aea11e1e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:37.092520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1365/2758] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:38:46.141910 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=72 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:47.078928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=26 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:38:55.798279 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1776a-0fdb-42d9-a61e-6c7c3e05f417 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:38:55.798754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1776a-0fdb-42d9-a61e-6c7c3e05f417 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:38:55.798840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1776a-0fdb-42d9-a61e-6c7c3e05f417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:38:55.817781 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-77f1776a-0fdb-42d9-a61e-6c7c3e05f417 None None] MFA Rules not processed for user `3be82c73106c48fa83ab08e96ecbfe81`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:38:55.819018 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-77f1776a-0fdb-42d9-a61e-6c7c3e05f417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:38:55.825715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1394/2759] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:38:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:39:05.812583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:39:22.691455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e197156-e66c-4aae-999c-d2395f2898e4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:22.691935 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e197156-e66c-4aae-999c-d2395f2898e4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:22.692025 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e197156-e66c-4aae-999c-d2395f2898e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:22.711194 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-0e197156-e66c-4aae-999c-d2395f2898e4 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:39:22.712380 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0e197156-e66c-4aae-999c-d2395f2898e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:22.767552 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1366/2760] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:39:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:39:22.807417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1776a-0fdb-42d9-a61e-6c7c3e05f417 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:22.808811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-77f1776a-0fdb-42d9-a61e-6c7c3e05f417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:22.877061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1776a-0fdb-42d9-a61e-6c7c3e05f417 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:22.881874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f93e94-3422-425c-b484-a2088a4b71b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:22.882488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f93e94-3422-425c-b484-a2088a4b71b3 service swift] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:22.882766 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f93e94-3422-425c-b484-a2088a4b71b3 service swift] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:22.883037 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f93e94-3422-425c-b484-a2088a4b71b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:22.890943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f93e94-3422-425c-b484-a2088a4b71b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:22.892577 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f93e94-3422-425c-b484-a2088a4b71b3 service swift] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:22.900749 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1395/2761] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:39:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:39:31.911289 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e197156-e66c-4aae-999c-d2395f2898e4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:31.912936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-0e197156-e66c-4aae-999c-d2395f2898e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:32.033907 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e197156-e66c-4aae-999c-d2395f2898e4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:32.039018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d333f71-21a6-4107-9aed-bb253a8ac087 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:32.039618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d333f71-21a6-4107-9aed-bb253a8ac087 service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:32.039908 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d333f71-21a6-4107-9aed-bb253a8ac087 service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:32.040173 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d333f71-21a6-4107-9aed-bb253a8ac087 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:32.041922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2d333f71-21a6-4107-9aed-bb253a8ac087 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:32.053298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d333f71-21a6-4107-9aed-bb253a8ac087 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:32.054882 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d333f71-21a6-4107-9aed-bb253a8ac087 service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:32.065493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1367/2762] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:39:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:39:32.897726 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=32 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:39:39.491003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61213f6f-f460-4f66-8af6-03ddd87140d9 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:39.491556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61213f6f-f460-4f66-8af6-03ddd87140d9 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:39.491556 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61213f6f-f460-4f66-8af6-03ddd87140d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:39.511034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-61213f6f-f460-4f66-8af6-03ddd87140d9 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:39:39.512223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-61213f6f-f460-4f66-8af6-03ddd87140d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:39.519717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1396/2763] 10.222.0.57 () {58 vars in 967 bytes} [Mon Aug 8 17:39:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:39:39.531489 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d333f71-21a6-4107-9aed-bb253a8ac087 service placement] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:39.543110 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d333f71-21a6-4107-9aed-bb253a8ac087 service placement] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:39.556419 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e40da-a75a-429b-b809-3b52bc4cf904 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:39.557018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e40da-a75a-429b-b809-3b52bc4cf904 service swift] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:39.557294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e40da-a75a-429b-b809-3b52bc4cf904 service swift] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:39.557562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e40da-a75a-429b-b809-3b52bc4cf904 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:39.566100 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e40da-a75a-429b-b809-3b52bc4cf904 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:39.567653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e40da-a75a-429b-b809-3b52bc4cf904 service swift] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:39.575542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1368/2764] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:39:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 17:39:42.446828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61213f6f-f460-4f66-8af6-03ddd87140d9 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:42.458594 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61213f6f-f460-4f66-8af6-03ddd87140d9 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:42.463183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:42.463811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:42.464087 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:42.464356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] PATH_INFO: `/v3/users/3be82c73106c48fa83ab08e96ecbfe81` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:42.465467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be82c73106c48fa83ab08e96ecbfe81)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:42.467412 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:42.619572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be82c73106c48fa83ab08e96ecbfe81'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:42.662813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be82c73106c48fa83ab08e96ecbfe81'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:42.669811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be82c73106c48fa83ab08e96ecbfe81'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:42.682522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be82c73106c48fa83ab08e96ecbfe81'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:42.683186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-d53c1f76-4cc6-42c4-b6a0-4d48bcabbd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be82c73106c48fa83ab08e96ecbfe81'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:42.689093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1397/2765] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:39:42 2022] DELETE /identity/v3/users/3be82c73106c48fa83ab08e96ecbfe81 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:39:42.921630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e40da-a75a-429b-b809-3b52bc4cf904 service swift] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:42.938172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e40da-a75a-429b-b809-3b52bc4cf904 service swift] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:42.969660 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:39:43.005573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:43.006221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.006497 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.006792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] PATH_INFO: `/v3/projects/2a87a1912d31432eaa11f28865ca7d98` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.007899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a87a1912d31432eaa11f28865ca7d98)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:43.009801 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:43.098721 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a87a1912d31432eaa11f28865ca7d98', 'request_id': 'req-b09b55ec-8977-4351-907f-dbabb17c4a41'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:43.104876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a87a1912d31432eaa11f28865ca7d98', 'request_id': 'req-b09b55ec-8977-4351-907f-dbabb17c4a41'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:43.166212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a87a1912d31432eaa11f28865ca7d98', 'request_id': 'req-b09b55ec-8977-4351-907f-dbabb17c4a41'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:43.177630 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] The token cache is being invalidate because project 2a87a1912d31432eaa11f28865ca7d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:39:43.177970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:43.178593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b09b55ec-8977-4351-907f-dbabb17c4a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:39:43.180096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1369/2766] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:39:42 2022] DELETE /identity/v3/projects/2a87a1912d31432eaa11f28865ca7d98 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:39:43.186013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507389d5-1144-4579-94a3-ba148198843e None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.186320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507389d5-1144-4579-94a3-ba148198843e None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.186593 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507389d5-1144-4579-94a3-ba148198843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.206300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-507389d5-1144-4579-94a3-ba148198843e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:39:43.207490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-507389d5-1144-4579-94a3-ba148198843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:43.264207 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1398/2767] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:39:43.272573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507ee42-085d-495d-a32e-bf66dbbbf9c4 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.272870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507ee42-085d-495d-a32e-bf66dbbbf9c4 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.273123 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507ee42-085d-495d-a32e-bf66dbbbf9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.292942 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c507ee42-085d-495d-a32e-bf66dbbbf9c4 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:39:43.294172 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c507ee42-085d-495d-a32e-bf66dbbbf9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:43.302702 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1370/2768] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:39:43.306712 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507389d5-1144-4579-94a3-ba148198843e None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:43.316353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507389d5-1144-4579-94a3-ba148198843e None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:43.321066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa90bd-4fef-476a-b73e-3dd5114aa258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:43.321655 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa90bd-4fef-476a-b73e-3dd5114aa258 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.321976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa90bd-4fef-476a-b73e-3dd5114aa258 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.322248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa90bd-4fef-476a-b73e-3dd5114aa258 admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.322886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa90bd-4fef-476a-b73e-3dd5114aa258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:39:43.323186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa90bd-4fef-476a-b73e-3dd5114aa258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:43.324815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa90bd-4fef-476a-b73e-3dd5114aa258 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:43.336098 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1399/2769] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:39:43.340408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507ee42-085d-495d-a32e-bf66dbbbf9c4 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:43.350565 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507ee42-085d-495d-a32e-bf66dbbbf9c4 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:43.355223 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6e62b-ab77-4e83-afd0-819185b5f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:43.355806 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6e62b-ab77-4e83-afd0-819185b5f66b admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.356084 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6e62b-ab77-4e83-afd0-819185b5f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.356356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6e62b-ab77-4e83-afd0-819185b5f66b admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.356980 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6e62b-ab77-4e83-afd0-819185b5f66b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:43.358937 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6e62b-ab77-4e83-afd0-819185b5f66b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:43.441291 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1371/2770] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:39:43 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:39:43.445298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa90bd-4fef-476a-b73e-3dd5114aa258 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:43.457413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa90bd-4fef-476a-b73e-3dd5114aa258 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:43.506943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:39:43.516126 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b06f69-02c2-4fb5-ad2a-58641fd5e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:43.516850 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b06f69-02c2-4fb5-ad2a-58641fd5e747 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.520287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b06f69-02c2-4fb5-ad2a-58641fd5e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.520287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b06f69-02c2-4fb5-ad2a-58641fd5e747 admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.520287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b06f69-02c2-4fb5-ad2a-58641fd5e747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:43.520287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b06f69-02c2-4fb5-ad2a-58641fd5e747 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:43.539920 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6e62b-ab77-4e83-afd0-819185b5f66b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:43.541667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-47e6e62b-ab77-4e83-afd0-819185b5f66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:43.599188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6e62b-ab77-4e83-afd0-819185b5f66b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:43.602885 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60585daa-fe9c-4efe-b7bc-820236522804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:43.603505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60585daa-fe9c-4efe-b7bc-820236522804 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.603839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60585daa-fe9c-4efe-b7bc-820236522804 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.604046 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60585daa-fe9c-4efe-b7bc-820236522804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.605879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-60585daa-fe9c-4efe-b7bc-820236522804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:43.609544 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1400/2771] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:39:43 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:39:43.610576 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b06f69-02c2-4fb5-ad2a-58641fd5e747 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:43.616321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60585daa-fe9c-4efe-b7bc-820236522804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:43.617750 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60585daa-fe9c-4efe-b7bc-820236522804 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:43.620606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b06f69-02c2-4fb5-ad2a-58641fd5e747 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:43.625276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb08d0-9f8f-45ca-995c-d49180f023da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:43.625887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb08d0-9f8f-45ca-995c-d49180f023da admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.626163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb08d0-9f8f-45ca-995c-d49180f023da admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.626434 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb08d0-9f8f-45ca-995c-d49180f023da admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.626633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1372/2772] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:39:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:39:43.627061 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb08d0-9f8f-45ca-995c-d49180f023da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:43.629135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb08d0-9f8f-45ca-995c-d49180f023da admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:43.641170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1401/2773] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:39:43.645755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60585daa-fe9c-4efe-b7bc-820236522804 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:43.657169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60585daa-fe9c-4efe-b7bc-820236522804 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:43.662460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c2d61-6ce0-48dc-a366-863d9d8e5668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:43.663040 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2d61-6ce0-48dc-a366-863d9d8e5668 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.663440 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2d61-6ce0-48dc-a366-863d9d8e5668 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.663719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2d61-6ce0-48dc-a366-863d9d8e5668 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.664355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c2d61-6ce0-48dc-a366-863d9d8e5668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:43.665977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c2d61-6ce0-48dc-a366-863d9d8e5668 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:43.676126 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1373/2774] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:39:43.680222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb08d0-9f8f-45ca-995c-d49180f023da admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:43.690151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb08d0-9f8f-45ca-995c-d49180f023da admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:43.694831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9640f-746c-42d8-9f80-f721712428be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:43.695415 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9640f-746c-42d8-9f80-f721712428be admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.695691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9640f-746c-42d8-9f80-f721712428be admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.695964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9640f-746c-42d8-9f80-f721712428be admin admin] PATH_INFO: `/v3/projects/16e69a5645214463a9c76a02903f0f3c/users/7f6251b0292243bf87df290fdaa5673b/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.710672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9640f-746c-42d8-9f80-f721712428be admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e69a5645214463a9c76a02903f0f3c, user_id=7f6251b0292243bf87df290fdaa5673b, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:43.713329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9640f-746c-42d8-9f80-f721712428be admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:43.777925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1402/2775] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:39:43 2022] PUT /identity/v3/projects/16e69a5645214463a9c76a02903f0f3c/users/7f6251b0292243bf87df290fdaa5673b/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:39:43.788569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab2674-b642-4737-8c25-c240fe37b3bf None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.790470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab2674-b642-4737-8c25-c240fe37b3bf None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.790470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab2674-b642-4737-8c25-c240fe37b3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.816366 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-38ab2674-b642-4737-8c25-c240fe37b3bf None None] MFA Rules not processed for user `7f6251b0292243bf87df290fdaa5673b`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:39:43.817607 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-38ab2674-b642-4737-8c25-c240fe37b3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:43.830817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:39:43.882382 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1374/2776] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:39:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:39:43.893676 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9640f-746c-42d8-9f80-f721712428be admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:39:43.895483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-63a9640f-746c-42d8-9f80-f721712428be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:43.952891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9640f-746c-42d8-9f80-f721712428be admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:39:43.956980 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3d4d7-3e20-425c-86c6-3d49b81fd722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:39:43.957586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3d4d7-3e20-425c-86c6-3d49b81fd722 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:43.957887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3d4d7-3e20-425c-86c6-3d49b81fd722 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:43.958167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3d4d7-3e20-425c-86c6-3d49b81fd722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:43.966854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3d4d7-3e20-425c-86c6-3d49b81fd722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:39:43.968308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3d4d7-3e20-425c-86c6-3d49b81fd722 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:39:43.978047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1403/2777] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:39:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:39:45.676711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30181e-f442-4d0a-9293-ca730549ac10 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:39:45.677174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30181e-f442-4d0a-9293-ca730549ac10 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:39:45.677258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30181e-f442-4d0a-9293-ca730549ac10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:39:45.696367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1b30181e-f442-4d0a-9293-ca730549ac10 None None] MFA Rules not processed for user `7f6251b0292243bf87df290fdaa5673b`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:39:45.697579 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1b30181e-f442-4d0a-9293-ca730549ac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:39:45.704257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1375/2778] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:39:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:39:53.974111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=46,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:39:55.691616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=23 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:02.457504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390a72c-6762-4abe-9211-bf7ecdd6da49 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:02.458119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390a72c-6762-4abe-9211-bf7ecdd6da49 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:02.458119 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390a72c-6762-4abe-9211-bf7ecdd6da49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:02.479806 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-3390a72c-6762-4abe-9211-bf7ecdd6da49 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:40:02.481045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-3390a72c-6762-4abe-9211-bf7ecdd6da49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:02.535813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1404/2779] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:40:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:40:02.548274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30181e-f442-4d0a-9293-ca730549ac10 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:02.549686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1b30181e-f442-4d0a-9293-ca730549ac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:02.622298 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30181e-f442-4d0a-9293-ca730549ac10 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:02.627152 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874bd8-6ad3-44c0-a771-2829fa527691 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:02.627742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874bd8-6ad3-44c0-a771-2829fa527691 service swift] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:02.628017 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874bd8-6ad3-44c0-a771-2829fa527691 service swift] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:02.628339 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874bd8-6ad3-44c0-a771-2829fa527691 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:02.636113 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3874bd8-6ad3-44c0-a771-2829fa527691 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:02.637673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3874bd8-6ad3-44c0-a771-2829fa527691 service swift] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:02.645559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1376/2780] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:40:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:40:03.521357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1895bb-9bde-406c-8f9a-e44bdf9cab7d None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:03.526734 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1895bb-9bde-406c-8f9a-e44bdf9cab7d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:03.526734 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1895bb-9bde-406c-8f9a-e44bdf9cab7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:03.541538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6f1895bb-9bde-406c-8f9a-e44bdf9cab7d None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:40:03.542824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6f1895bb-9bde-406c-8f9a-e44bdf9cab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:03.550337 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1405/2781] 10.222.0.57 () {58 vars in 967 bytes} [Mon Aug 8 17:40:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:40:03.563182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874bd8-6ad3-44c0-a771-2829fa527691 service swift] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:03.573957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874bd8-6ad3-44c0-a771-2829fa527691 service swift] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:03.579108 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5a962-8aea-4674-a401-7df5516d2153 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:03.579809 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5a962-8aea-4674-a401-7df5516d2153 service swift] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:03.580091 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5a962-8aea-4674-a401-7df5516d2153 service swift] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:03.580374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5a962-8aea-4674-a401-7df5516d2153 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:03.594193 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5a962-8aea-4674-a401-7df5516d2153 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:03.595779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5a962-8aea-4674-a401-7df5516d2153 service swift] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:03.605039 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1377/2782] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:40:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:40:04.187709 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae864fc-1034-4ee7-bf7a-ca77205e023c None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:04.188026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae864fc-1034-4ee7-bf7a-ca77205e023c None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:04.188295 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae864fc-1034-4ee7-bf7a-ca77205e023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:04.207504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-aae864fc-1034-4ee7-bf7a-ca77205e023c None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:40:04.208642 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-aae864fc-1034-4ee7-bf7a-ca77205e023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:04.216663 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1406/2783] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Aug 8 17:40:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:40:04.228634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5a962-8aea-4674-a401-7df5516d2153 service swift] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:04.239105 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5a962-8aea-4674-a401-7df5516d2153 service swift] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:04.244092 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb2eee-5588-4fe1-86d7-445810a5fdf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:04.244677 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb2eee-5588-4fe1-86d7-445810a5fdf7 service swift] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:04.244950 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb2eee-5588-4fe1-86d7-445810a5fdf7 service swift] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:04.245217 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb2eee-5588-4fe1-86d7-445810a5fdf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:04.252969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb2eee-5588-4fe1-86d7-445810a5fdf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:04.254570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb2eee-5588-4fe1-86d7-445810a5fdf7 service swift] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:04.263133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1378/2784] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:40:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:40:04.830785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df618822-f222-4142-84a5-a3b3956a0214 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:04.831300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df618822-f222-4142-84a5-a3b3956a0214 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:04.831391 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df618822-f222-4142-84a5-a3b3956a0214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:04.850232 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-df618822-f222-4142-84a5-a3b3956a0214 None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:40:04.851441 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-df618822-f222-4142-84a5-a3b3956a0214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:04.859026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1407/2785] 10.222.0.57 () {58 vars in 967 bytes} [Mon Aug 8 17:40:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:40:04.871218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb2eee-5588-4fe1-86d7-445810a5fdf7 service swift] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:04.881573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb2eee-5588-4fe1-86d7-445810a5fdf7 service swift] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:04.887412 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac6855-d69b-44f3-8a3c-53e4d20172cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:04.888050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac6855-d69b-44f3-8a3c-53e4d20172cc service swift] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:04.888331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac6855-d69b-44f3-8a3c-53e4d20172cc service swift] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:04.888596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac6855-d69b-44f3-8a3c-53e4d20172cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:04.897164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac6855-d69b-44f3-8a3c-53e4d20172cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:04.898988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac6855-d69b-44f3-8a3c-53e4d20172cc service swift] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:04.907482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1379/2786] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:40:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:40:07.662585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df618822-f222-4142-84a5-a3b3956a0214 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:07.724270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df618822-f222-4142-84a5-a3b3956a0214 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:07.729287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:07.729935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:07.730212 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:07.730482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] PATH_INFO: `/v3/users/7f6251b0292243bf87df290fdaa5673b` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:07.731592 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6251b0292243bf87df290fdaa5673b)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:07.733473 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:07.947168 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6251b0292243bf87df290fdaa5673b'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.022399 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6251b0292243bf87df290fdaa5673b'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.030745 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6251b0292243bf87df290fdaa5673b'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.042912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6251b0292243bf87df290fdaa5673b'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.043554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6251b0292243bf87df290fdaa5673b'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.049364 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1408/2787] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:40:07 2022] DELETE /identity/v3/users/7f6251b0292243bf87df290fdaa5673b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:40:08.053898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ea947-b201-4c89-9c0d-78a22917dbec None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:08.054667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ea947-b201-4c89-9c0d-78a22917dbec None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:08.054667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ea947-b201-4c89-9c0d-78a22917dbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:08.074742 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-591ea947-b201-4c89-9c0d-78a22917dbec None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:40:08.075923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-591ea947-b201-4c89-9c0d-78a22917dbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:08.134937 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1380/2788] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:40:08.146366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:08.148561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:08.155568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:08.206717 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0abd-785e-43b7-909c-77da98bbef60 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:08.210318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6549be-c55b-4dc6-8122-d2a5be7cd9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:08.211542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6549be-c55b-4dc6-8122-d2a5be7cd9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:08.212029 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6549be-c55b-4dc6-8122-d2a5be7cd9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:08.212545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6549be-c55b-4dc6-8122-d2a5be7cd9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:08.222670 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6549be-c55b-4dc6-8122-d2a5be7cd9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:08.224335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6549be-c55b-4dc6-8122-d2a5be7cd9b9 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:08.234886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1409/2789] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:40:08.366186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9ed3e-85fe-4e38-b13f-c4c8e71690be None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:08.367067 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9ed3e-85fe-4e38-b13f-c4c8e71690be None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:08.367345 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9ed3e-85fe-4e38-b13f-c4c8e71690be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:08.386621 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-7ba9ed3e-85fe-4e38-b13f-c4c8e71690be None None] MFA Rules not processed for user `19cf8bdae18e46ac8217c021765a2a78`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:40:08.387736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7ba9ed3e-85fe-4e38-b13f-c4c8e71690be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:08.434036 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=100 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:08.444165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1381/2790] 10.222.0.57 () {58 vars in 967 bytes} [Mon Aug 8 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:40:08.457641 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6549be-c55b-4dc6-8122-d2a5be7cd9b9 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:08.459052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ce6549be-c55b-4dc6-8122-d2a5be7cd9b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:08.470993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9ed3e-85fe-4e38-b13f-c4c8e71690be None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:08.472681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7ba9ed3e-85fe-4e38-b13f-c4c8e71690be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:08.488617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9ed3e-85fe-4e38-b13f-c4c8e71690be None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:08.494601 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:08.495326 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:08.495682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:08.496013 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] PATH_INFO: `/v3/projects/16e69a5645214463a9c76a02903f0f3c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:08.497626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e69a5645214463a9c76a02903f0f3c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:08.499640 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:08.540867 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6549be-c55b-4dc6-8122-d2a5be7cd9b9 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:08.546276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fea3b9-57f8-4ffc-90a9-90d730e0f86a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92130daa3c3a49c2adc597dc563b2973', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:08.546943 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fea3b9-57f8-4ffc-90a9-90d730e0f86a service swift] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:08.547219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fea3b9-57f8-4ffc-90a9-90d730e0f86a service swift] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:08.547492 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fea3b9-57f8-4ffc-90a9-90d730e0f86a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:08.557978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fea3b9-57f8-4ffc-90a9-90d730e0f86a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:08.559561 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fea3b9-57f8-4ffc-90a9-90d730e0f86a service swift] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:08.568247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1410/2791] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Aug 8 17:40:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 17:40:08.591634 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e69a5645214463a9c76a02903f0f3c', 'request_id': 'req-3a6be979-7dd2-4e56-bd99-5d2932ab718c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.598430 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e69a5645214463a9c76a02903f0f3c', 'request_id': 'req-3a6be979-7dd2-4e56-bd99-5d2932ab718c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.709429 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e69a5645214463a9c76a02903f0f3c', 'request_id': 'req-3a6be979-7dd2-4e56-bd99-5d2932ab718c'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.720858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] The token cache is being invalidate because project 16e69a5645214463a9c76a02903f0f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:40:08.721163 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.721831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:08.724470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1382/2792] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:40:08 2022] DELETE /identity/v3/projects/16e69a5645214463a9c76a02903f0f3c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:40:12.799268 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fea3b9-57f8-4ffc-90a9-90d730e0f86a service swift] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:12.805586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-78fea3b9-57f8-4ffc-90a9-90d730e0f86a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:12.871046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fea3b9-57f8-4ffc-90a9-90d730e0f86a service swift] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:12.908074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:12.931468 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8335b-20cd-483b-868b-da3fa108a72f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:12.932228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8335b-20cd-483b-868b-da3fa108a72f service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:12.932386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8335b-20cd-483b-868b-da3fa108a72f service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:12.932662 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8335b-20cd-483b-868b-da3fa108a72f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:12.934497 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ecd8335b-20cd-483b-868b-da3fa108a72f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:12.947970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8335b-20cd-483b-868b-da3fa108a72f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:12.949586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8335b-20cd-483b-868b-da3fa108a72f service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:12.964698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1411/2793] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:40:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:40:16.603891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:16.607732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:16.671931 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be979-7dd2-4e56-bd99-5d2932ab718c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:16.676367 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:16.676976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:16.677246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:16.677509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] PATH_INFO: `/v3/users/640d1ecc20f54d81945ba6003787ebc7` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:16.678629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=640d1ecc20f54d81945ba6003787ebc7)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:16.680515 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:16.876723 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640d1ecc20f54d81945ba6003787ebc7'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:16.937246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640d1ecc20f54d81945ba6003787ebc7'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:16.949949 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640d1ecc20f54d81945ba6003787ebc7'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:16.957021 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640d1ecc20f54d81945ba6003787ebc7'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:16.957690 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640d1ecc20f54d81945ba6003787ebc7'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:16.959246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:16.964147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1383/2794] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:40:16 2022] DELETE /identity/v3/users/640d1ecc20f54d81945ba6003787ebc7 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:40:17.242219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8335b-20cd-483b-868b-da3fa108a72f service placement] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:17.253265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8335b-20cd-483b-868b-da3fa108a72f service placement] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:17.312611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:17.313240 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:17.313586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:17.313897 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] PATH_INFO: `/v3/projects/f8b25b8f45c84feb869633ea4ee98521` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:17.315019 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b25b8f45c84feb869633ea4ee98521)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:17.316917 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:17.418250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b25b8f45c84feb869633ea4ee98521', 'request_id': 'req-95a3cbc7-959c-483c-9429-300d6d462cb6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.424515 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b25b8f45c84feb869633ea4ee98521', 'request_id': 'req-95a3cbc7-959c-483c-9429-300d6d462cb6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.497192 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b25b8f45c84feb869633ea4ee98521', 'request_id': 'req-95a3cbc7-959c-483c-9429-300d6d462cb6'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.507657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] The token cache is being invalidate because project f8b25b8f45c84feb869633ea4ee98521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:40:17.507938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.508551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.510085 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1412/2795] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:40:17 2022] DELETE /identity/v3/projects/f8b25b8f45c84feb869633ea4ee98521 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:40:17.514854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:17.516252 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:17.529162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39deab-586d-496b-972b-6a33f9f35287 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:17.584785 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:17.585410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:17.585683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:17.585977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] PATH_INFO: `/v3/users/0bbea4fe1fb04d099269f7e93833765e` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:17.587098 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bbea4fe1fb04d099269f7e93833765e)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:17.589029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:17.762860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bbea4fe1fb04d099269f7e93833765e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.829233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bbea4fe1fb04d099269f7e93833765e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.841676 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bbea4fe1fb04d099269f7e93833765e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.848575 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bbea4fe1fb04d099269f7e93833765e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.849195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-62be22bd-944e-41b2-8d4b-93e64d4aaa7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bbea4fe1fb04d099269f7e93833765e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:17.855560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1384/2796] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:40:17 2022] DELETE /identity/v3/users/0bbea4fe1fb04d099269f7e93833765e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:40:18.106026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:18.107490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:18.121114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3cbc7-959c-483c-9429-300d6d462cb6 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:18.177578 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:18.178236 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:18.178514 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:18.178787 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] PATH_INFO: `/v3/projects/29bafbef2b8640d7b626ff304f52af86` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:18.179870 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29bafbef2b8640d7b626ff304f52af86)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:18.181760 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:18.183204 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:18.313926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29bafbef2b8640d7b626ff304f52af86', 'request_id': 'req-920e78d5-6122-4dd8-8554-f7023af96b16'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:18.320131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29bafbef2b8640d7b626ff304f52af86', 'request_id': 'req-920e78d5-6122-4dd8-8554-f7023af96b16'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:18.393741 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29bafbef2b8640d7b626ff304f52af86', 'request_id': 'req-920e78d5-6122-4dd8-8554-f7023af96b16'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:18.404915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] The token cache is being invalidate because project 29bafbef2b8640d7b626ff304f52af86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:40:18.405202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:18.405850 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:40:18.407327 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1413/2797] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:40:18 2022] DELETE /identity/v3/projects/29bafbef2b8640d7b626ff304f52af86 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:40:27.851624 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:28.401199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:36.519346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed5003-0479-4286-92dc-a8d56882d5c1 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:36.519809 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed5003-0479-4286-92dc-a8d56882d5c1 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:36.519899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed5003-0479-4286-92dc-a8d56882d5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:36.538539 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-7fed5003-0479-4286-92dc-a8d56882d5c1 None None] MFA Rules not processed for user `57662358169d4830b2ebb7d594efb701`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:40:36.539730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7fed5003-0479-4286-92dc-a8d56882d5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:36.597166 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1385/2798] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:40:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:40:38.888491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:40:38.890005 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:38.967569 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e78d5-6122-4dd8-8554-f7023af96b16 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:40:38.972382 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:40:38.976116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:38.976116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:38.976116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:38.976116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:39.060324 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:40:39.061915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:40:39.082617 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1414/2799] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:40:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:40:46.587602 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=25 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:49.080104 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=70 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:40:50.677501 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20bf9d-4a39-49e3-93c8-b273792e6d0e None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:40:50.678029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20bf9d-4a39-49e3-93c8-b273792e6d0e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:40:50.678118 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20bf9d-4a39-49e3-93c8-b273792e6d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:40:50.698819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-9a20bf9d-4a39-49e3-93c8-b273792e6d0e None None] MFA Rules not processed for user `054f2a730eaa4a32b11f4a731c687094`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:40:50.700009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9a20bf9d-4a39-49e3-93c8-b273792e6d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:40:50.760779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1386/2800] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:40:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:41:00.753981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=25 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:09.912691 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:09.914034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:09.927510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c4e9-9e80-4f66-b1d7-cebe9f3e3daa service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:09.932879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:09.933462 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:09.933738 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:09.934034 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:09.935677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:09.945375 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:09.946942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:09.955549 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1415/2801] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:18.672629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20bf9d-4a39-49e3-93c8-b273792e6d0e None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:18.682919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20bf9d-4a39-49e3-93c8-b273792e6d0e None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:18.686410 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd9284-0aa8-4410-b3ce-4d0b47b43c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:18.686994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd9284-0aa8-4410-b3ce-4d0b47b43c70 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:18.687269 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd9284-0aa8-4410-b3ce-4d0b47b43c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:18.687536 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd9284-0aa8-4410-b3ce-4d0b47b43c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:18.689230 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b7fd9284-0aa8-4410-b3ce-4d0b47b43c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:18.750700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd9284-0aa8-4410-b3ce-4d0b47b43c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:18.752112 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd9284-0aa8-4410-b3ce-4d0b47b43c70 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:18.761442 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1387/2802] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:19.951608 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=6 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:22.379023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:22.380585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:22.395502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09711f1-3821-4c2f-8218-8b7001f14f97 service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:22.400906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:22.400906 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:22.401108 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:22.401277 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] PATH_INFO: `/v3/users/054f2a730eaa4a32b11f4a731c687094` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:22.402502 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=054f2a730eaa4a32b11f4a731c687094)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:22.404512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:22.606182 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054f2a730eaa4a32b11f4a731c687094'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:22.669345 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054f2a730eaa4a32b11f4a731c687094'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:22.676496 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054f2a730eaa4a32b11f4a731c687094'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:22.688276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054f2a730eaa4a32b11f4a731c687094'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:22.688925 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5ca5422b-149e-48ac-bbc4-3f36ebbc1214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054f2a730eaa4a32b11f4a731c687094'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:22.694722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1416/2803] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:42:22 2022] DELETE /identity/v3/users/054f2a730eaa4a32b11f4a731c687094 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:42:22.953567 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd9284-0aa8-4410-b3ce-4d0b47b43c70 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:22.964054 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd9284-0aa8-4410-b3ce-4d0b47b43c70 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:23.020779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:23.021396 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.021674 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.021996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] PATH_INFO: `/v3/projects/e449345a18254a1ab181cbbcfafa33c8` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.023720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e449345a18254a1ab181cbbcfafa33c8)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:23.025600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:23.163682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e449345a18254a1ab181cbbcfafa33c8', 'request_id': 'req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:23.170622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e449345a18254a1ab181cbbcfafa33c8', 'request_id': 'req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:23.241244 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e449345a18254a1ab181cbbcfafa33c8', 'request_id': 'req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:23.251870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] The token cache is being invalidate because project e449345a18254a1ab181cbbcfafa33c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:42:23.252153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:23.252767 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-6ba5e8c3-f075-49c4-8627-7feeb62a0cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:23.254296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1388/2804] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:42:22 2022] DELETE /identity/v3/projects/e449345a18254a1ab181cbbcfafa33c8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:42:23.259847 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b8e51-147e-4747-ac25-be2ae53eac5b None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.269161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b8e51-147e-4747-ac25-be2ae53eac5b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.269161 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b8e51-147e-4747-ac25-be2ae53eac5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.279799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-e95b8e51-147e-4747-ac25-be2ae53eac5b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:23.281109 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-e95b8e51-147e-4747-ac25-be2ae53eac5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:23.338707 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1417/2805] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:23.346462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160f9e5-4286-4338-950b-332082e378f0 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.347041 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160f9e5-4286-4338-950b-332082e378f0 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.347311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160f9e5-4286-4338-950b-332082e378f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.367218 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-6160f9e5-4286-4338-950b-332082e378f0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:23.368450 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-6160f9e5-4286-4338-950b-332082e378f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:23.376336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1389/2806] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:23.380329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b8e51-147e-4747-ac25-be2ae53eac5b None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:23.390508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b8e51-147e-4747-ac25-be2ae53eac5b None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:23.395261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c572de7-0029-4040-a2f0-a72071db219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:23.395859 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c572de7-0029-4040-a2f0-a72071db219b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.396138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c572de7-0029-4040-a2f0-a72071db219b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.396410 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c572de7-0029-4040-a2f0-a72071db219b admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.397053 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c572de7-0029-4040-a2f0-a72071db219b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:42:23.397354 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c572de7-0029-4040-a2f0-a72071db219b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:23.398999 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c572de7-0029-4040-a2f0-a72071db219b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:23.410483 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1418/2807] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:42:23.414650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160f9e5-4286-4338-950b-332082e378f0 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:23.423700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160f9e5-4286-4338-950b-332082e378f0 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:23.428823 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4e1c4-3d3d-49a2-8851-aadde667ae0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:23.429402 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4e1c4-3d3d-49a2-8851-aadde667ae0f admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.429678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4e1c4-3d3d-49a2-8851-aadde667ae0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.429981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4e1c4-3d3d-49a2-8851-aadde667ae0f admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.430599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4e1c4-3d3d-49a2-8851-aadde667ae0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:23.432475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4e1c4-3d3d-49a2-8851-aadde667ae0f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:23.448127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:23.576824 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1390/2808] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:42:23 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:23.580860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c572de7-0029-4040-a2f0-a72071db219b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:23.591101 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c572de7-0029-4040-a2f0-a72071db219b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:23.646355 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cf152-52ed-4a39-8669-499ff4255d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:23.646959 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cf152-52ed-4a39-8669-499ff4255d3e admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.647245 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cf152-52ed-4a39-8669-499ff4255d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.647515 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cf152-52ed-4a39-8669-499ff4255d3e admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.648137 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cf152-52ed-4a39-8669-499ff4255d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:23.650060 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cf152-52ed-4a39-8669-499ff4255d3e admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:23.657686 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:23.757255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1419/2809] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:42:23 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:42:23.761328 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4e1c4-3d3d-49a2-8851-aadde667ae0f admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:23.771858 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4e1c4-3d3d-49a2-8851-aadde667ae0f admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:23.776480 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6b26c-46bb-4d2e-a734-4c16c75022d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:23.777066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6b26c-46bb-4d2e-a734-4c16c75022d0 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.777340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6b26c-46bb-4d2e-a734-4c16c75022d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.777605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6b26c-46bb-4d2e-a734-4c16c75022d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.778268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6b26c-46bb-4d2e-a734-4c16c75022d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:23.779850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6b26c-46bb-4d2e-a734-4c16c75022d0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:23.790150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1391/2810] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:23.793887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cf152-52ed-4a39-8669-499ff4255d3e admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:23.804241 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cf152-52ed-4a39-8669-499ff4255d3e admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:23.808824 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1d87c-edf8-4c66-8af9-6f3a652cf6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:23.815754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1d87c-edf8-4c66-8af9-6f3a652cf6bf admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.815754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1d87c-edf8-4c66-8af9-6f3a652cf6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.815754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1d87c-edf8-4c66-8af9-6f3a652cf6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.815754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1d87c-edf8-4c66-8af9-6f3a652cf6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:23.815754 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1d87c-edf8-4c66-8af9-6f3a652cf6bf admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:23.822733 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1420/2811] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:23.826617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6b26c-46bb-4d2e-a734-4c16c75022d0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:23.836531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6b26c-46bb-4d2e-a734-4c16c75022d0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:23.841139 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21a518-cf30-453d-88e1-c672620e2666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:23.841763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21a518-cf30-453d-88e1-c672620e2666 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.842064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21a518-cf30-453d-88e1-c672620e2666 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.842330 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21a518-cf30-453d-88e1-c672620e2666 admin admin] PATH_INFO: `/v3/projects/7d9ba0c94d074d39bfeb68867a4dfcc0/users/5ce8c0f6a0dc4568a53d0cb193996901/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.858845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21a518-cf30-453d-88e1-c672620e2666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9ba0c94d074d39bfeb68867a4dfcc0, user_id=5ce8c0f6a0dc4568a53d0cb193996901, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:23.861508 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21a518-cf30-453d-88e1-c672620e2666 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:23.926090 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1392/2812] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:23 2022] PUT /identity/v3/projects/7d9ba0c94d074d39bfeb68867a4dfcc0/users/5ce8c0f6a0dc4568a53d0cb193996901/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:23.937912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163160d1-fc5e-45fb-94a9-4f0629221dbe None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:23.938219 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163160d1-fc5e-45fb-94a9-4f0629221dbe None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:23.938479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163160d1-fc5e-45fb-94a9-4f0629221dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:23.965852 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-163160d1-fc5e-45fb-94a9-4f0629221dbe None None] MFA Rules not processed for user `5ce8c0f6a0dc4568a53d0cb193996901`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:23.967088 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-163160d1-fc5e-45fb-94a9-4f0629221dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:24.030815 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1421/2813] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:24.035237 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21a518-cf30-453d-88e1-c672620e2666 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:24.095600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21a518-cf30-453d-88e1-c672620e2666 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:24.101226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700e738-4c35-4f61-99b2-0b55b5927b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:24.102133 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700e738-4c35-4f61-99b2-0b55b5927b08 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:24.102413 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700e738-4c35-4f61-99b2-0b55b5927b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:24.102680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700e738-4c35-4f61-99b2-0b55b5927b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:24.103299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700e738-4c35-4f61-99b2-0b55b5927b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:24.105179 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700e738-4c35-4f61-99b2-0b55b5927b08 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:24.190205 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1393/2814] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:42:24 2022] POST /identity/v3/projects => generated 363 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:24.194288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163160d1-fc5e-45fb-94a9-4f0629221dbe None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:24.205685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163160d1-fc5e-45fb-94a9-4f0629221dbe None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:24.262810 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb520b-c339-4160-bb7e-134f30d479ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:24.263413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb520b-c339-4160-bb7e-134f30d479ed admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:24.263692 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb520b-c339-4160-bb7e-134f30d479ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:24.263989 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb520b-c339-4160-bb7e-134f30d479ed admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:24.264615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb520b-c339-4160-bb7e-134f30d479ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:24.266596 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb520b-c339-4160-bb7e-134f30d479ed admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:24.345482 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1422/2815] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:42:24 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:24.349700 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700e738-4c35-4f61-99b2-0b55b5927b08 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:24.360421 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700e738-4c35-4f61-99b2-0b55b5927b08 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:24.364998 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc5805-87cf-4e07-abb1-2084617a60c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:24.365582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc5805-87cf-4e07-abb1-2084617a60c9 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:24.365951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc5805-87cf-4e07-abb1-2084617a60c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:24.366221 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc5805-87cf-4e07-abb1-2084617a60c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:24.366861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc5805-87cf-4e07-abb1-2084617a60c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:24.368448 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc5805-87cf-4e07-abb1-2084617a60c9 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:24.381967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1394/2816] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:24.386303 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb520b-c339-4160-bb7e-134f30d479ed admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:24.396294 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb520b-c339-4160-bb7e-134f30d479ed admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:24.401570 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b7c97-955b-4ae3-a1ae-55cf54049fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:24.402246 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b7c97-955b-4ae3-a1ae-55cf54049fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:24.402522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b7c97-955b-4ae3-a1ae-55cf54049fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:24.402790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b7c97-955b-4ae3-a1ae-55cf54049fc2 admin admin] PATH_INFO: `/v3/projects/19a83fe9e0c943aba7c594306af15edc/users/fad0d47ca3d448feab53ce6bde0114b1/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:24.417895 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b7c97-955b-4ae3-a1ae-55cf54049fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a83fe9e0c943aba7c594306af15edc, user_id=fad0d47ca3d448feab53ce6bde0114b1, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:24.420548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b7c97-955b-4ae3-a1ae-55cf54049fc2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:24.488378 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1423/2817] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:24 2022] PUT /identity/v3/projects/19a83fe9e0c943aba7c594306af15edc/users/fad0d47ca3d448feab53ce6bde0114b1/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:24.492354 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc5805-87cf-4e07-abb1-2084617a60c9 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:24.502360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc5805-87cf-4e07-abb1-2084617a60c9 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:24.557239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fd372-ff6e-445f-86d8-79d7b311986c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:24.557867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fd372-ff6e-445f-86d8-79d7b311986c admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:24.558148 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fd372-ff6e-445f-86d8-79d7b311986c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:24.558416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fd372-ff6e-445f-86d8-79d7b311986c admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:24.559044 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fd372-ff6e-445f-86d8-79d7b311986c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:24.560619 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fd372-ff6e-445f-86d8-79d7b311986c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:24.570636 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1395/2818] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:24.574985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b7c97-955b-4ae3-a1ae-55cf54049fc2 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:24.586433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b7c97-955b-4ae3-a1ae-55cf54049fc2 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:24.591978 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b5628-feb0-479b-96c0-8de58c52c381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:24.592647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b5628-feb0-479b-96c0-8de58c52c381 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:24.592951 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b5628-feb0-479b-96c0-8de58c52c381 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:24.593828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b5628-feb0-479b-96c0-8de58c52c381 admin admin] PATH_INFO: `/v3/projects/19a83fe9e0c943aba7c594306af15edc/users/fad0d47ca3d448feab53ce6bde0114b1/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:24.595366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b5628-feb0-479b-96c0-8de58c52c381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a83fe9e0c943aba7c594306af15edc, user_id=fad0d47ca3d448feab53ce6bde0114b1, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:24.598442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b5628-feb0-479b-96c0-8de58c52c381 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:24.610404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1424/2819] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:24 2022] PUT /identity/v3/projects/19a83fe9e0c943aba7c594306af15edc/users/fad0d47ca3d448feab53ce6bde0114b1/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:24.620023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c0eb8-27d9-4b45-915f-25b91c4edb6c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:24.620315 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c0eb8-27d9-4b45-915f-25b91c4edb6c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:24.620617 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c0eb8-27d9-4b45-915f-25b91c4edb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:24.650729 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-e98c0eb8-27d9-4b45-915f-25b91c4edb6c None None] MFA Rules not processed for user `fad0d47ca3d448feab53ce6bde0114b1`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:24.651924 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e98c0eb8-27d9-4b45-915f-25b91c4edb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:24.672559 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:24.725838 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1396/2820] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:24.738222 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b5628-feb0-479b-96c0-8de58c52c381 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:24.740178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-419b5628-feb0-479b-96c0-8de58c52c381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:24.764542 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:24.800374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b5628-feb0-479b-96c0-8de58c52c381 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:24.804742 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4edca1-7a0d-42b5-87cc-e714fc7d6f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:24.805360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4edca1-7a0d-42b5-87cc-e714fc7d6f9f service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:24.805715 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4edca1-7a0d-42b5-87cc-e714fc7d6f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:24.806028 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4edca1-7a0d-42b5-87cc-e714fc7d6f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:24.815797 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4edca1-7a0d-42b5-87cc-e714fc7d6f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:24.817255 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4edca1-7a0d-42b5-87cc-e714fc7d6f9f service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:24.833985 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1425/2821] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:25.756475 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c0eb8-27d9-4b45-915f-25b91c4edb6c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:25.816053 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c0eb8-27d9-4b45-915f-25b91c4edb6c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:25.821315 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:25.821961 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:25.822233 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:25.822493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] PATH_INFO: `/v3/users/5ce8c0f6a0dc4568a53d0cb193996901` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:25.823595 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce8c0f6a0dc4568a53d0cb193996901)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:25.825483 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:26.042886 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce8c0f6a0dc4568a53d0cb193996901'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:26.149280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce8c0f6a0dc4568a53d0cb193996901'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:26.162050 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce8c0f6a0dc4568a53d0cb193996901'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:26.169305 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce8c0f6a0dc4568a53d0cb193996901'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:26.169985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce8c0f6a0dc4568a53d0cb193996901'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:26.175793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1397/2822] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:42:25 2022] DELETE /identity/v3/users/5ce8c0f6a0dc4568a53d0cb193996901 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:26.181084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389b831-4f82-4371-a252-02211b2d32a1 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:26.181532 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389b831-4f82-4371-a252-02211b2d32a1 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:26.181615 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389b831-4f82-4371-a252-02211b2d32a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:26.200178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-f389b831-4f82-4371-a252-02211b2d32a1 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:26.201363 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f389b831-4f82-4371-a252-02211b2d32a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:26.258155 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1426/2823] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:42:26.267876 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:26.269308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:26.310520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:26.326638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209cfc1-1557-4bfc-95c4-92cdf549a422 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:26.330216 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab275bb1-196e-4d0b-80fa-e2e3ad434a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:26.330804 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab275bb1-196e-4d0b-80fa-e2e3ad434a94 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:26.331080 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab275bb1-196e-4d0b-80fa-e2e3ad434a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:26.331349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab275bb1-196e-4d0b-80fa-e2e3ad434a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:26.341159 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab275bb1-196e-4d0b-80fa-e2e3ad434a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:26.342599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab275bb1-196e-4d0b-80fa-e2e3ad434a94 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:26.352732 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1398/2824] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:26.784679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389b831-4f82-4371-a252-02211b2d32a1 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:26.786131 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f389b831-4f82-4371-a252-02211b2d32a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:26.798299 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389b831-4f82-4371-a252-02211b2d32a1 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:26.802936 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:26.803564 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:26.803844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:26.804111 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] PATH_INFO: `/v3/projects/7d9ba0c94d074d39bfeb68867a4dfcc0` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:26.805248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9ba0c94d074d39bfeb68867a4dfcc0)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:26.807183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:26.926466 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9ba0c94d074d39bfeb68867a4dfcc0', 'request_id': 'req-83b73f8d-23ad-4679-a4f0-11da54275faa'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:26.933065 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9ba0c94d074d39bfeb68867a4dfcc0', 'request_id': 'req-83b73f8d-23ad-4679-a4f0-11da54275faa'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:27.042949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9ba0c94d074d39bfeb68867a4dfcc0', 'request_id': 'req-83b73f8d-23ad-4679-a4f0-11da54275faa'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:27.054026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] The token cache is being invalidate because project 7d9ba0c94d074d39bfeb68867a4dfcc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:42:27.054312 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:27.055431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:27.056940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1427/2825] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:42:26 2022] DELETE /identity/v3/projects/7d9ba0c94d074d39bfeb68867a4dfcc0 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:42:27.060861 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab275bb1-196e-4d0b-80fa-e2e3ad434a94 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:27.071308 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab275bb1-196e-4d0b-80fa-e2e3ad434a94 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:27.126174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:27.126792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:27.127069 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:27.127336 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] PATH_INFO: `/v3/users/fad0d47ca3d448feab53ce6bde0114b1` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:27.128432 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad0d47ca3d448feab53ce6bde0114b1)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:27.130380 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:27.322420 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad0d47ca3d448feab53ce6bde0114b1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:27.383103 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad0d47ca3d448feab53ce6bde0114b1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:27.395351 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad0d47ca3d448feab53ce6bde0114b1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:27.402278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad0d47ca3d448feab53ce6bde0114b1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:27.402922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-2c7d237d-1431-43b5-96c1-c671999a6768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad0d47ca3d448feab53ce6bde0114b1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:27.408645 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1399/2826] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:42:27 2022] DELETE /identity/v3/users/fad0d47ca3d448feab53ce6bde0114b1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:27.821713 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:27.823174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:27.837140 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73f8d-23ad-4679-a4f0-11da54275faa admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:27.882510 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:27.894675 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:27.895308 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:27.895587 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:27.895863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] PATH_INFO: `/v3/projects/19a83fe9e0c943aba7c594306af15edc` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:27.896972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a83fe9e0c943aba7c594306af15edc)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:27.899539 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:28.198135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a83fe9e0c943aba7c594306af15edc', 'request_id': 'req-be10c61e-d7d0-4dc1-9234-ccf924f97e43'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:28.204353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a83fe9e0c943aba7c594306af15edc', 'request_id': 'req-be10c61e-d7d0-4dc1-9234-ccf924f97e43'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:28.248073 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a83fe9e0c943aba7c594306af15edc', 'request_id': 'req-be10c61e-d7d0-4dc1-9234-ccf924f97e43'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:28.258225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] The token cache is being invalidate because project 19a83fe9e0c943aba7c594306af15edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:42:28.258509 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:28.259790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-be10c61e-d7d0-4dc1-9234-ccf924f97e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:28.261266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1428/2827] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:42:27 2022] DELETE /identity/v3/projects/19a83fe9e0c943aba7c594306af15edc => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:28.266657 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc07e0-5aea-4043-8419-d2e09589fff0 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:28.267140 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc07e0-5aea-4043-8419-d2e09589fff0 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:28.267294 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc07e0-5aea-4043-8419-d2e09589fff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:28.287015 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-59dc07e0-5aea-4043-8419-d2e09589fff0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:28.288188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-59dc07e0-5aea-4043-8419-d2e09589fff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:28.336859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:28.346859 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1400/2828] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:28.355793 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393470f-25a7-406a-bd65-68c047d1f746 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:28.356078 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393470f-25a7-406a-bd65-68c047d1f746 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:28.356351 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393470f-25a7-406a-bd65-68c047d1f746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:28.376855 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-8393470f-25a7-406a-bd65-68c047d1f746 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:28.381151 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8393470f-25a7-406a-bd65-68c047d1f746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:28.386680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1429/2829] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:28.390926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc07e0-5aea-4043-8419-d2e09589fff0 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:28.401116 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc07e0-5aea-4043-8419-d2e09589fff0 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:28.405651 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9a10d-25dc-48f4-abd6-1a341bdc71e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:28.409923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9a10d-25dc-48f4-abd6-1a341bdc71e5 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:28.409923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9a10d-25dc-48f4-abd6-1a341bdc71e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:28.409923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9a10d-25dc-48f4-abd6-1a341bdc71e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:28.409923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9a10d-25dc-48f4-abd6-1a341bdc71e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:42:28.409923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9a10d-25dc-48f4-abd6-1a341bdc71e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:28.409923 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9a10d-25dc-48f4-abd6-1a341bdc71e5 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:28.422062 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1401/2830] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:42:28.426410 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393470f-25a7-406a-bd65-68c047d1f746 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:28.436287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393470f-25a7-406a-bd65-68c047d1f746 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:28.441043 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1e125-099b-406a-be25-969730d9722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:28.441638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1e125-099b-406a-be25-969730d9722f admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:28.441960 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1e125-099b-406a-be25-969730d9722f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:28.442234 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1e125-099b-406a-be25-969730d9722f admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:28.442856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1e125-099b-406a-be25-969730d9722f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:28.444794 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1e125-099b-406a-be25-969730d9722f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:28.550421 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1430/2831] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:42:28 2022] POST /identity/v3/projects => generated 357 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:28.554653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9a10d-25dc-48f4-abd6-1a341bdc71e5 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:28.565107 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9a10d-25dc-48f4-abd6-1a341bdc71e5 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:28.622465 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db0986-f5cb-4d9a-9f5d-69b158db2c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:28.623064 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db0986-f5cb-4d9a-9f5d-69b158db2c6e admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:28.623341 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db0986-f5cb-4d9a-9f5d-69b158db2c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:28.623608 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db0986-f5cb-4d9a-9f5d-69b158db2c6e admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:28.624226 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db0986-f5cb-4d9a-9f5d-69b158db2c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:28.626158 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db0986-f5cb-4d9a-9f5d-69b158db2c6e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:28.738057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1402/2832] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:42:28 2022] POST /identity/v3/users => generated 279 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:28.742320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1e125-099b-406a-be25-969730d9722f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:28.753444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1e125-099b-406a-be25-969730d9722f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:28.757972 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980865cd-87ec-4fb5-94a9-d99129017912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:28.758568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980865cd-87ec-4fb5-94a9-d99129017912 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:28.758869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980865cd-87ec-4fb5-94a9-d99129017912 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:28.759790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980865cd-87ec-4fb5-94a9-d99129017912 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:28.760426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980865cd-87ec-4fb5-94a9-d99129017912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:28.762040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980865cd-87ec-4fb5-94a9-d99129017912 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:28.771941 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1431/2833] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:28.775681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db0986-f5cb-4d9a-9f5d-69b158db2c6e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:28.785443 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db0986-f5cb-4d9a-9f5d-69b158db2c6e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:28.790699 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9468896-0e99-46d3-91e9-add9ec941259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:28.791279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9468896-0e99-46d3-91e9-add9ec941259 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:28.791553 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9468896-0e99-46d3-91e9-add9ec941259 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:28.791817 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9468896-0e99-46d3-91e9-add9ec941259 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:28.792449 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9468896-0e99-46d3-91e9-add9ec941259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:28.794136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9468896-0e99-46d3-91e9-add9ec941259 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:28.803928 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1403/2834] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:28.808479 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980865cd-87ec-4fb5-94a9-d99129017912 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:28.818196 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980865cd-87ec-4fb5-94a9-d99129017912 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:28.822645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77aa76-6f36-4c35-82ea-e41c5f9dfa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:28.823288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77aa76-6f36-4c35-82ea-e41c5f9dfa1b admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:28.823527 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77aa76-6f36-4c35-82ea-e41c5f9dfa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:28.823798 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77aa76-6f36-4c35-82ea-e41c5f9dfa1b admin admin] PATH_INFO: `/v3/projects/f168c794e45446e189130d8f7e4e3c87/users/34bbeab91c8d4f0fbb3ff7f9da81d5ce/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:28.837890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77aa76-6f36-4c35-82ea-e41c5f9dfa1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f168c794e45446e189130d8f7e4e3c87, user_id=34bbeab91c8d4f0fbb3ff7f9da81d5ce, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:28.840506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77aa76-6f36-4c35-82ea-e41c5f9dfa1b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:28.932218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1432/2835] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:28 2022] PUT /identity/v3/projects/f168c794e45446e189130d8f7e4e3c87/users/34bbeab91c8d4f0fbb3ff7f9da81d5ce/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:28.942362 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608ea36-c7c1-481b-be71-790317b6e330 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:28.942650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608ea36-c7c1-481b-be71-790317b6e330 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:28.942925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608ea36-c7c1-481b-be71-790317b6e330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:28.969691 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-3608ea36-c7c1-481b-be71-790317b6e330 None None] MFA Rules not processed for user `34bbeab91c8d4f0fbb3ff7f9da81d5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:28.970981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-3608ea36-c7c1-481b-be71-790317b6e330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:29.035314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1404/2836] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:29.039883 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77aa76-6f36-4c35-82ea-e41c5f9dfa1b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:29.103844 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77aa76-6f36-4c35-82ea-e41c5f9dfa1b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:29.108661 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8379d-2f33-461b-9909-654012cc510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:29.109257 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8379d-2f33-461b-9909-654012cc510b admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:29.109529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8379d-2f33-461b-9909-654012cc510b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:29.109813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8379d-2f33-461b-9909-654012cc510b admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:29.110426 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8379d-2f33-461b-9909-654012cc510b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:29.112293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8379d-2f33-461b-9909-654012cc510b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:29.185880 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:29.192888 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1433/2837] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:42:29 2022] POST /identity/v3/projects => generated 357 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:29.196914 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608ea36-c7c1-481b-be71-790317b6e330 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:29.207721 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608ea36-c7c1-481b-be71-790317b6e330 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:29.254384 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:29.263004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f93ea6-0f6a-416d-a76d-4b759061a4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:29.263631 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f93ea6-0f6a-416d-a76d-4b759061a4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:29.263904 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f93ea6-0f6a-416d-a76d-4b759061a4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:29.264209 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f93ea6-0f6a-416d-a76d-4b759061a4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:29.264782 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f93ea6-0f6a-416d-a76d-4b759061a4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:29.266709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f93ea6-0f6a-416d-a76d-4b759061a4a4 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:29.340210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1405/2838] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:42:29 2022] POST /identity/v3/users => generated 285 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:29.344092 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8379d-2f33-461b-9909-654012cc510b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:29.355499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8379d-2f33-461b-9909-654012cc510b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:29.360128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a8af8-e68f-4b87-95b9-adae9aef1ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:29.360719 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a8af8-e68f-4b87-95b9-adae9aef1ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:29.360997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a8af8-e68f-4b87-95b9-adae9aef1ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:29.361266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a8af8-e68f-4b87-95b9-adae9aef1ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:29.361969 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a8af8-e68f-4b87-95b9-adae9aef1ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:29.363554 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a8af8-e68f-4b87-95b9-adae9aef1ad5 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:29.373946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1434/2839] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:29.378174 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f93ea6-0f6a-416d-a76d-4b759061a4a4 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:29.388088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f93ea6-0f6a-416d-a76d-4b759061a4a4 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:29.392825 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9acd78-643a-438a-8853-88402956f7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:29.393460 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9acd78-643a-438a-8853-88402956f7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:29.393737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9acd78-643a-438a-8853-88402956f7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:29.394049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9acd78-643a-438a-8853-88402956f7f7 admin admin] PATH_INFO: `/v3/projects/6d8eecb8b55941a9881d8e31687bc804/users/e63d8cb8c3734ac7b790b57e040bbd16/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:29.409683 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9acd78-643a-438a-8853-88402956f7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8eecb8b55941a9881d8e31687bc804, user_id=e63d8cb8c3734ac7b790b57e040bbd16, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:29.412503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9acd78-643a-438a-8853-88402956f7f7 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:29.470897 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1406/2840] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:29 2022] PUT /identity/v3/projects/6d8eecb8b55941a9881d8e31687bc804/users/e63d8cb8c3734ac7b790b57e040bbd16/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:29.475189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a8af8-e68f-4b87-95b9-adae9aef1ad5 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:29.485624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a8af8-e68f-4b87-95b9-adae9aef1ad5 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:29.541128 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404d03e-bb79-431d-9e4e-199823a69d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:29.541729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404d03e-bb79-431d-9e4e-199823a69d57 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:29.542035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404d03e-bb79-431d-9e4e-199823a69d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:29.542306 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404d03e-bb79-431d-9e4e-199823a69d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:29.542942 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404d03e-bb79-431d-9e4e-199823a69d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:29.544528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404d03e-bb79-431d-9e4e-199823a69d57 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:29.554178 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1435/2841] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:29.558563 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9acd78-643a-438a-8853-88402956f7f7 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:29.568820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9acd78-643a-438a-8853-88402956f7f7 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:29.573978 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0055f-7f9c-4c82-a5f2-6e9b376bf7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:29.574611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0055f-7f9c-4c82-a5f2-6e9b376bf7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:29.574890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0055f-7f9c-4c82-a5f2-6e9b376bf7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:29.575160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0055f-7f9c-4c82-a5f2-6e9b376bf7d2 admin admin] PATH_INFO: `/v3/projects/6d8eecb8b55941a9881d8e31687bc804/users/e63d8cb8c3734ac7b790b57e040bbd16/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:29.577072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0055f-7f9c-4c82-a5f2-6e9b376bf7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8eecb8b55941a9881d8e31687bc804, user_id=e63d8cb8c3734ac7b790b57e040bbd16, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:29.579737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0055f-7f9c-4c82-a5f2-6e9b376bf7d2 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:29.591505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1407/2842] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:29 2022] PUT /identity/v3/projects/6d8eecb8b55941a9881d8e31687bc804/users/e63d8cb8c3734ac7b790b57e040bbd16/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:29.601202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f0355-fd20-45bb-9ed3-7635930f03db None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:29.601486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f0355-fd20-45bb-9ed3-7635930f03db None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:29.601755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f0355-fd20-45bb-9ed3-7635930f03db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:29.628450 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-1d5f0355-fd20-45bb-9ed3-7635930f03db None None] MFA Rules not processed for user `e63d8cb8c3734ac7b790b57e040bbd16`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:29.629650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-1d5f0355-fd20-45bb-9ed3-7635930f03db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:29.695887 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1436/2843] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:29.720719 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0055f-7f9c-4c82-a5f2-6e9b376bf7d2 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:29.722695 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5fc0055f-7f9c-4c82-a5f2-6e9b376bf7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:29.778457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0055f-7f9c-4c82-a5f2-6e9b376bf7d2 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:29.782061 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9eb81-d9c2-47ca-bbf8-401a4258e1b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:29.782643 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9eb81-d9c2-47ca-bbf8-401a4258e1b2 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:29.782919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9eb81-d9c2-47ca-bbf8-401a4258e1b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:29.783188 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9eb81-d9c2-47ca-bbf8-401a4258e1b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:29.791917 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9eb81-d9c2-47ca-bbf8-401a4258e1b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:29.793334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9eb81-d9c2-47ca-bbf8-401a4258e1b2 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:29.803275 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1408/2844] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:29.964405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f0355-fd20-45bb-9ed3-7635930f03db None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:29.974135 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f0355-fd20-45bb-9ed3-7635930f03db None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:29.977621 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d851e-4a33-47d2-ae33-989c98f39f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:29.978249 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d851e-4a33-47d2-ae33-989c98f39f32 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:29.978598 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d851e-4a33-47d2-ae33-989c98f39f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:29.978873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d851e-4a33-47d2-ae33-989c98f39f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:30.034984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d851e-4a33-47d2-ae33-989c98f39f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:30.036437 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d851e-4a33-47d2-ae33-989c98f39f32 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:30.045493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1437/2845] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:33.568178 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9eb81-d9c2-47ca-bbf8-401a4258e1b2 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:33.575759 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9eb81-d9c2-47ca-bbf8-401a4258e1b2 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:33.579355 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d476e-650d-49bb-8fc3-73e1d8492e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:33.579948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d476e-650d-49bb-8fc3-73e1d8492e46 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:33.580219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d476e-650d-49bb-8fc3-73e1d8492e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:33.580482 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d476e-650d-49bb-8fc3-73e1d8492e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:33.582234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-529d476e-650d-49bb-8fc3-73e1d8492e46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:33.646562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d476e-650d-49bb-8fc3-73e1d8492e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:33.647971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d476e-650d-49bb-8fc3-73e1d8492e46 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:33.657323 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1409/2846] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:40.041413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:42.097874 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cfaee-aaf8-475a-a29c-570e1d08f31f None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:42.098486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cfaee-aaf8-475a-a29c-570e1d08f31f None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:42.098486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cfaee-aaf8-475a-a29c-570e1d08f31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:42.117114 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-992cfaee-aaf8-475a-a29c-570e1d08f31f None None] MFA Rules not processed for user `57662358169d4830b2ebb7d594efb701`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:42.118335 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-992cfaee-aaf8-475a-a29c-570e1d08f31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:42.125475 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1438/2847] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:43.653185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:51.879198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d476e-650d-49bb-8fc3-73e1d8492e46 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:51.939512 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d476e-650d-49bb-8fc3-73e1d8492e46 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:51.943867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:51.944472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:51.944744 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:51.945009 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] PATH_INFO: `/v3/users/34bbeab91c8d4f0fbb3ff7f9da81d5ce` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:51.946136 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34bbeab91c8d4f0fbb3ff7f9da81d5ce)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:51.948019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:52.111704 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:52.133334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34bbeab91c8d4f0fbb3ff7f9da81d5ce'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:52.230881 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34bbeab91c8d4f0fbb3ff7f9da81d5ce'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:52.243599 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34bbeab91c8d4f0fbb3ff7f9da81d5ce'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:52.250277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34bbeab91c8d4f0fbb3ff7f9da81d5ce'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:52.250926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34bbeab91c8d4f0fbb3ff7f9da81d5ce'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:52.256598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1410/2848] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:42:51 2022] DELETE /identity/v3/users/34bbeab91c8d4f0fbb3ff7f9da81d5ce => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:52.260966 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16b7af-b3d4-474b-887e-0b765de5801d None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:52.261276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16b7af-b3d4-474b-887e-0b765de5801d None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:52.261548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16b7af-b3d4-474b-887e-0b765de5801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:52.280651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-2c16b7af-b3d4-474b-887e-0b765de5801d None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:52.281857 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2c16b7af-b3d4-474b-887e-0b765de5801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:52.343116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1439/2849] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:52.352799 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:52.354321 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:52.418812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12fa2c-d67e-4c85-891f-16cd8db87d25 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:52.422428 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d88371-fea4-4f04-9e0e-b98a3db222ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:52.423023 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d88371-fea4-4f04-9e0e-b98a3db222ca service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:52.423296 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d88371-fea4-4f04-9e0e-b98a3db222ca service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:52.423562 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d88371-fea4-4f04-9e0e-b98a3db222ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:52.432401 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d88371-fea4-4f04-9e0e-b98a3db222ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:52.433808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d88371-fea4-4f04-9e0e-b98a3db222ca service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:52.443236 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1411/2850] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:52.847396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16b7af-b3d4-474b-887e-0b765de5801d None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:52.847396 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2c16b7af-b3d4-474b-887e-0b765de5801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:52.852401 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16b7af-b3d4-474b-887e-0b765de5801d None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:52.856975 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:52.857602 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:52.858540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:52.858819 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] PATH_INFO: `/v3/projects/f168c794e45446e189130d8f7e4e3c87` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:52.860202 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f168c794e45446e189130d8f7e4e3c87)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:52.862026 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:52.977640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f168c794e45446e189130d8f7e4e3c87', 'request_id': 'req-099b2893-f999-4101-965f-bb979f83175c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:52.983751 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f168c794e45446e189130d8f7e4e3c87', 'request_id': 'req-099b2893-f999-4101-965f-bb979f83175c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.047949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f168c794e45446e189130d8f7e4e3c87', 'request_id': 'req-099b2893-f999-4101-965f-bb979f83175c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.058499 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] The token cache is being invalidate because project f168c794e45446e189130d8f7e4e3c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:42:53.058785 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.059403 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.070402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1440/2851] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:42:52 2022] DELETE /identity/v3/projects/f168c794e45446e189130d8f7e4e3c87 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:53.070540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d88371-fea4-4f04-9e0e-b98a3db222ca service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:53.075088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d88371-fea4-4f04-9e0e-b98a3db222ca service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:53.083844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:53.130375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:53.130985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:53.131258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:53.131521 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] PATH_INFO: `/v3/users/e63d8cb8c3734ac7b790b57e040bbd16` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:53.132610 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63d8cb8c3734ac7b790b57e040bbd16)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:53.134665 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:53.329670 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63d8cb8c3734ac7b790b57e040bbd16'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.374280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63d8cb8c3734ac7b790b57e040bbd16'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.387401 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63d8cb8c3734ac7b790b57e040bbd16'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.394035 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63d8cb8c3734ac7b790b57e040bbd16'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.394658 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-79763ec1-2c80-49e8-8555-c8945fcf7567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63d8cb8c3734ac7b790b57e040bbd16'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.400360 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1412/2852] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:42:53 2022] DELETE /identity/v3/users/e63d8cb8c3734ac7b790b57e040bbd16 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:53.784991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:53.786585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:53.800611 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2893-f999-4101-965f-bb979f83175c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:53.858003 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:53.858640 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:53.858920 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:53.859189 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] PATH_INFO: `/v3/projects/6d8eecb8b55941a9881d8e31687bc804` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:53.860385 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8eecb8b55941a9881d8e31687bc804)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:53.862384 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:53.947265 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:53.963392 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8eecb8b55941a9881d8e31687bc804', 'request_id': 'req-7a4b8336-1393-41d7-a45e-5d18dabecfe2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:53.970116 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8eecb8b55941a9881d8e31687bc804', 'request_id': 'req-7a4b8336-1393-41d7-a45e-5d18dabecfe2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:54.048414 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8eecb8b55941a9881d8e31687bc804', 'request_id': 'req-7a4b8336-1393-41d7-a45e-5d18dabecfe2'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:54.059639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] The token cache is being invalidate because project 6d8eecb8b55941a9881d8e31687bc804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:42:54.059991 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:54.060624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-7a4b8336-1393-41d7-a45e-5d18dabecfe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:54.062179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1441/2853] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:42:53 2022] DELETE /identity/v3/projects/6d8eecb8b55941a9881d8e31687bc804 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:54.067185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260c09-c2ea-4d4c-89b3-7104062da07c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.067472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260c09-c2ea-4d4c-89b3-7104062da07c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.067743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260c09-c2ea-4d4c-89b3-7104062da07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.087364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a3260c09-c2ea-4d4c-89b3-7104062da07c None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:54.088589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a3260c09-c2ea-4d4c-89b3-7104062da07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:54.149255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1413/2854] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:42:54.157922 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d17a0c-fd76-459b-8b4a-c5b268e28684 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.158258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d17a0c-fd76-459b-8b4a-c5b268e28684 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.158537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d17a0c-fd76-459b-8b4a-c5b268e28684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.178645 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-d7d17a0c-fd76-459b-8b4a-c5b268e28684 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:54.179842 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-d7d17a0c-fd76-459b-8b4a-c5b268e28684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:54.188208 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1442/2855] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:54.192471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3260c09-c2ea-4d4c-89b3-7104062da07c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:54.202573 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3260c09-c2ea-4d4c-89b3-7104062da07c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:54.207121 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f4bec-8132-4e6b-8601-4cec0d3e2133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:54.207705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f4bec-8132-4e6b-8601-4cec0d3e2133 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.207981 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f4bec-8132-4e6b-8601-4cec0d3e2133 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.208249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f4bec-8132-4e6b-8601-4cec0d3e2133 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.208887 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f4bec-8132-4e6b-8601-4cec0d3e2133 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:42:54.209185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f4bec-8132-4e6b-8601-4cec0d3e2133 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:54.210798 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f4bec-8132-4e6b-8601-4cec0d3e2133 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:54.222331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1414/2856] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:42:54.226470 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d17a0c-fd76-459b-8b4a-c5b268e28684 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:54.235449 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d17a0c-fd76-459b-8b4a-c5b268e28684 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:54.240022 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d917a3-368f-487e-b5ec-1bf046115ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:54.240609 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d917a3-368f-487e-b5ec-1bf046115ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.240885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d917a3-368f-487e-b5ec-1bf046115ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.241154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d917a3-368f-487e-b5ec-1bf046115ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.241770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d917a3-368f-487e-b5ec-1bf046115ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:54.243664 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d917a3-368f-487e-b5ec-1bf046115ab1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:54.306624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1443/2857] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:42:54 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:42:54.310293 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f4bec-8132-4e6b-8601-4cec0d3e2133 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:54.320625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f4bec-8132-4e6b-8601-4cec0d3e2133 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:54.337813 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:54.376704 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f636a6-a8f3-4b42-8290-c1f081ad41f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:54.377310 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f636a6-a8f3-4b42-8290-c1f081ad41f9 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.377586 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f636a6-a8f3-4b42-8290-c1f081ad41f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.377890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f636a6-a8f3-4b42-8290-c1f081ad41f9 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.378548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f636a6-a8f3-4b42-8290-c1f081ad41f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:54.381252 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f636a6-a8f3-4b42-8290-c1f081ad41f9 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:54.468147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1415/2858] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:42:54 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:42:54.469490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d917a3-368f-487e-b5ec-1bf046115ab1 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:54.480777 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d917a3-368f-487e-b5ec-1bf046115ab1 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:54.485340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab8341-1009-4442-adcf-4966da3ea088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:54.485976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab8341-1009-4442-adcf-4966da3ea088 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.486256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab8341-1009-4442-adcf-4966da3ea088 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.486522 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab8341-1009-4442-adcf-4966da3ea088 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.487186 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab8341-1009-4442-adcf-4966da3ea088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:54.488773 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab8341-1009-4442-adcf-4966da3ea088 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:54.499753 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1444/2859] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:54.503807 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f636a6-a8f3-4b42-8290-c1f081ad41f9 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:54.513778 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f636a6-a8f3-4b42-8290-c1f081ad41f9 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:54.518894 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ac693-1064-4da6-bce0-3140085eb8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:54.519507 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ac693-1064-4da6-bce0-3140085eb8da admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.519778 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ac693-1064-4da6-bce0-3140085eb8da admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.520058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ac693-1064-4da6-bce0-3140085eb8da admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.520730 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ac693-1064-4da6-bce0-3140085eb8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:54.522333 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ac693-1064-4da6-bce0-3140085eb8da admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:54.532870 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1416/2860] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:54.537047 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab8341-1009-4442-adcf-4966da3ea088 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:54.548325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab8341-1009-4442-adcf-4966da3ea088 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:54.552984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7df54-495d-4e29-a632-adfcd92c9a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:54.553649 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7df54-495d-4e29-a632-adfcd92c9a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.553973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7df54-495d-4e29-a632-adfcd92c9a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.554239 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7df54-495d-4e29-a632-adfcd92c9a17 admin admin] PATH_INFO: `/v3/projects/c03773c9482a407fae821af9f3aa4654/users/aeed708fac4d4f359919934cc5c90bb4/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.569630 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7df54-495d-4e29-a632-adfcd92c9a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03773c9482a407fae821af9f3aa4654, user_id=aeed708fac4d4f359919934cc5c90bb4, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:54.572359 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7df54-495d-4e29-a632-adfcd92c9a17 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:54.675533 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1445/2861] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:54 2022] PUT /identity/v3/projects/c03773c9482a407fae821af9f3aa4654/users/aeed708fac4d4f359919934cc5c90bb4/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:42:54.685576 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a01462-98d1-43ce-b08a-337c5dacd71e None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.686535 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a01462-98d1-43ce-b08a-337c5dacd71e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.686812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a01462-98d1-43ce-b08a-337c5dacd71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.713834 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-b3a01462-98d1-43ce-b08a-337c5dacd71e None None] MFA Rules not processed for user `aeed708fac4d4f359919934cc5c90bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:54.714996 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-b3a01462-98d1-43ce-b08a-337c5dacd71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:54.778618 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1417/2862] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:54.783046 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7df54-495d-4e29-a632-adfcd92c9a17 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:54.842568 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7df54-495d-4e29-a632-adfcd92c9a17 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:54.847425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbac3d-2e79-4252-948e-19e2fe6fce85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:54.848012 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbac3d-2e79-4252-948e-19e2fe6fce85 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:54.848287 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbac3d-2e79-4252-948e-19e2fe6fce85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:54.848552 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbac3d-2e79-4252-948e-19e2fe6fce85 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:54.849153 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbac3d-2e79-4252-948e-19e2fe6fce85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:54.851154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbac3d-2e79-4252-948e-19e2fe6fce85 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:54.944251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1446/2863] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:42:54 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:54.948455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a01462-98d1-43ce-b08a-337c5dacd71e None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:54.959249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a01462-98d1-43ce-b08a-337c5dacd71e None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:55.015461 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f3d1e-32e1-4fad-a332-a16e536a4a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:55.016055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f3d1e-32e1-4fad-a332-a16e536a4a9b admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:55.016331 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f3d1e-32e1-4fad-a332-a16e536a4a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:55.016600 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f3d1e-32e1-4fad-a332-a16e536a4a9b admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:55.017222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f3d1e-32e1-4fad-a332-a16e536a4a9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:55.019195 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f3d1e-32e1-4fad-a332-a16e536a4a9b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:55.103441 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1418/2864] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:42:54 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:55.104666 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbac3d-2e79-4252-948e-19e2fe6fce85 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:55.116041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbac3d-2e79-4252-948e-19e2fe6fce85 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:55.120976 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd1151-b469-40d3-9019-6f7464d6c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:55.121574 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd1151-b469-40d3-9019-6f7464d6c23a admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:55.121909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd1151-b469-40d3-9019-6f7464d6c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:55.123360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd1151-b469-40d3-9019-6f7464d6c23a admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:55.123360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd1151-b469-40d3-9019-6f7464d6c23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:55.124477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd1151-b469-40d3-9019-6f7464d6c23a admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:55.135581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1447/2865] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:55.139977 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f3d1e-32e1-4fad-a332-a16e536a4a9b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:55.149871 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f3d1e-32e1-4fad-a332-a16e536a4a9b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:55.155191 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ecd0f-25d0-47ad-995b-8b9f47d17055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:55.155845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ecd0f-25d0-47ad-995b-8b9f47d17055 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:55.156164 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ecd0f-25d0-47ad-995b-8b9f47d17055 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:55.156393 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ecd0f-25d0-47ad-995b-8b9f47d17055 admin admin] PATH_INFO: `/v3/projects/f83557e50e9a4c5aa8774ee63bdc2e6f/users/63f96724cf564de9a4cf5ce61fa13fd4/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:55.172040 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ecd0f-25d0-47ad-995b-8b9f47d17055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f83557e50e9a4c5aa8774ee63bdc2e6f, user_id=63f96724cf564de9a4cf5ce61fa13fd4, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:55.174792 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ecd0f-25d0-47ad-995b-8b9f47d17055 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:55.177132 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:55.271839 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1419/2866] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:55 2022] PUT /identity/v3/projects/f83557e50e9a4c5aa8774ee63bdc2e6f/users/63f96724cf564de9a4cf5ce61fa13fd4/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:55.275679 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd1151-b469-40d3-9019-6f7464d6c23a admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:55.285477 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd1151-b469-40d3-9019-6f7464d6c23a admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:55.305004 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:55.340627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723406d6-d526-453f-89ef-e231f2ba4d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:55.341225 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723406d6-d526-453f-89ef-e231f2ba4d8f admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:55.341506 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723406d6-d526-453f-89ef-e231f2ba4d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:55.341775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723406d6-d526-453f-89ef-e231f2ba4d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:55.342438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723406d6-d526-453f-89ef-e231f2ba4d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:55.344015 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723406d6-d526-453f-89ef-e231f2ba4d8f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:55.353723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1448/2867] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:55.358353 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ecd0f-25d0-47ad-995b-8b9f47d17055 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:55.368530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ecd0f-25d0-47ad-995b-8b9f47d17055 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:55.373096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a9b68-b10d-457f-b1a4-b2afc9bebfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:55.374395 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a9b68-b10d-457f-b1a4-b2afc9bebfe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:55.374669 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a9b68-b10d-457f-b1a4-b2afc9bebfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:55.374934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a9b68-b10d-457f-b1a4-b2afc9bebfe8 admin admin] PATH_INFO: `/v3/projects/f83557e50e9a4c5aa8774ee63bdc2e6f/users/63f96724cf564de9a4cf5ce61fa13fd4/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:55.376820 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a9b68-b10d-457f-b1a4-b2afc9bebfe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f83557e50e9a4c5aa8774ee63bdc2e6f, user_id=63f96724cf564de9a4cf5ce61fa13fd4, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:55.379454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a9b68-b10d-457f-b1a4-b2afc9bebfe8 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:55.391570 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1420/2868] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:55 2022] PUT /identity/v3/projects/f83557e50e9a4c5aa8774ee63bdc2e6f/users/63f96724cf564de9a4cf5ce61fa13fd4/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:55.403132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38403a7f-773a-416b-845b-ba306f05c693 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:55.403429 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38403a7f-773a-416b-845b-ba306f05c693 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:55.403703 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38403a7f-773a-416b-845b-ba306f05c693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:55.431261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-38403a7f-773a-416b-845b-ba306f05c693 None None] MFA Rules not processed for user `63f96724cf564de9a4cf5ce61fa13fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:55.433682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-38403a7f-773a-416b-845b-ba306f05c693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:55.500191 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1449/2869] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:55.511901 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a9b68-b10d-457f-b1a4-b2afc9bebfe8 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:55.513812 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-188a9b68-b10d-457f-b1a4-b2afc9bebfe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:55.570462 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a9b68-b10d-457f-b1a4-b2afc9bebfe8 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:55.574037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6d164-7e0f-46d7-a16d-0272e5c1fb9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:55.574625 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d164-7e0f-46d7-a16d-0272e5c1fb9e service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:55.574900 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d164-7e0f-46d7-a16d-0272e5c1fb9e service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:55.575167 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d164-7e0f-46d7-a16d-0272e5c1fb9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:55.584086 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6d164-7e0f-46d7-a16d-0272e5c1fb9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:55.585519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6d164-7e0f-46d7-a16d-0272e5c1fb9e service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:55.595170 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1421/2870] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:55.662086 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38403a7f-773a-416b-845b-ba306f05c693 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:55.723997 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38403a7f-773a-416b-845b-ba306f05c693 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:55.728634 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:55.729260 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:55.729547 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:55.729846 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] PATH_INFO: `/v3/users/aeed708fac4d4f359919934cc5c90bb4` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:55.730987 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeed708fac4d4f359919934cc5c90bb4)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:55.733529 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:55.905280 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeed708fac4d4f359919934cc5c90bb4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:55.906916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:56.011469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeed708fac4d4f359919934cc5c90bb4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:56.018647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeed708fac4d4f359919934cc5c90bb4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:56.030888 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeed708fac4d4f359919934cc5c90bb4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:56.031521 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeed708fac4d4f359919934cc5c90bb4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:56.037984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1450/2871] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:42:55 2022] DELETE /identity/v3/users/aeed708fac4d4f359919934cc5c90bb4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:56.042596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df264e-0135-4610-a9d8-e0ae515fa1a0 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:56.042880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df264e-0135-4610-a9d8-e0ae515fa1a0 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:56.043155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df264e-0135-4610-a9d8-e0ae515fa1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:56.062364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-88df264e-0135-4610-a9d8-e0ae515fa1a0 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:56.063650 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-88df264e-0135-4610-a9d8-e0ae515fa1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:56.122679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1422/2872] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:56.132474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:56.133904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:56.191243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b957cb-82e6-4b13-9b9d-3fefb74329aa admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:56.194904 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97b4ec-1cc9-4973-9cff-eea9f9be1988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:56.195512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97b4ec-1cc9-4973-9cff-eea9f9be1988 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:56.195790 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97b4ec-1cc9-4973-9cff-eea9f9be1988 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:56.196058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97b4ec-1cc9-4973-9cff-eea9f9be1988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:56.206156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97b4ec-1cc9-4973-9cff-eea9f9be1988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:56.207720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97b4ec-1cc9-4973-9cff-eea9f9be1988 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:56.220461 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1451/2873] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:42:56.620936 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df264e-0135-4610-a9d8-e0ae515fa1a0 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:56.622437 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-88df264e-0135-4610-a9d8-e0ae515fa1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:56.634970 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df264e-0135-4610-a9d8-e0ae515fa1a0 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:56.640370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:56.641058 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:56.641342 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:56.641605 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] PATH_INFO: `/v3/projects/c03773c9482a407fae821af9f3aa4654` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:56.642755 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c03773c9482a407fae821af9f3aa4654)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:56.644713 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:56.751417 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c03773c9482a407fae821af9f3aa4654', 'request_id': 'req-1e507d88-9790-49a1-9a72-608c609d685e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:56.757681 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c03773c9482a407fae821af9f3aa4654', 'request_id': 'req-1e507d88-9790-49a1-9a72-608c609d685e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:56.812728 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c03773c9482a407fae821af9f3aa4654', 'request_id': 'req-1e507d88-9790-49a1-9a72-608c609d685e'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:56.824234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] The token cache is being invalidate because project c03773c9482a407fae821af9f3aa4654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:42:56.824638 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:56.825287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:56.827598 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1423/2874] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:42:56 2022] DELETE /identity/v3/projects/c03773c9482a407fae821af9f3aa4654 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:56.831430 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97b4ec-1cc9-4973-9cff-eea9f9be1988 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:56.841633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97b4ec-1cc9-4973-9cff-eea9f9be1988 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:56.899224 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:56.899860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:56.900141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:56.900411 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] PATH_INFO: `/v3/users/63f96724cf564de9a4cf5ce61fa13fd4` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:56.901537 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63f96724cf564de9a4cf5ce61fa13fd4)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:56.903651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:57.097165 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63f96724cf564de9a4cf5ce61fa13fd4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.190528 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63f96724cf564de9a4cf5ce61fa13fd4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.197840 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63f96724cf564de9a4cf5ce61fa13fd4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.203595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:57.210404 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63f96724cf564de9a4cf5ce61fa13fd4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.211049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-f7093fa5-b359-40f3-afb0-48f010e668f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63f96724cf564de9a4cf5ce61fa13fd4'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.216786 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1452/2875] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:42:56 2022] DELETE /identity/v3/users/63f96724cf564de9a4cf5ce61fa13fd4 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:42:57.649769 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:57.651274 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:57.665124 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507d88-9790-49a1-9a72-608c609d685e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:57.669757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:57.726668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:57.727287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:57.728548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:57.728548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] PATH_INFO: `/v3/projects/f83557e50e9a4c5aa8774ee63bdc2e6f` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:57.728982 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f83557e50e9a4c5aa8774ee63bdc2e6f)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:57.730899 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:57.845001 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f83557e50e9a4c5aa8774ee63bdc2e6f', 'request_id': 'req-859d82a8-f868-480c-9efd-55bc82e8f2ae'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.851363 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f83557e50e9a4c5aa8774ee63bdc2e6f', 'request_id': 'req-859d82a8-f868-480c-9efd-55bc82e8f2ae'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.939056 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f83557e50e9a4c5aa8774ee63bdc2e6f', 'request_id': 'req-859d82a8-f868-480c-9efd-55bc82e8f2ae'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.949285 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] The token cache is being invalidate because project f83557e50e9a4c5aa8774ee63bdc2e6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:42:57.949571 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.950219 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-859d82a8-f868-480c-9efd-55bc82e8f2ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:42:57.951714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1424/2876] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:42:57 2022] DELETE /identity/v3/projects/f83557e50e9a4c5aa8774ee63bdc2e6f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:57.956627 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7da466-5d3d-4801-b4bc-873ff7856207 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:57.956907 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7da466-5d3d-4801-b4bc-873ff7856207 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:57.957177 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7da466-5d3d-4801-b4bc-873ff7856207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:57.977486 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-da7da466-5d3d-4801-b4bc-873ff7856207 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:57.978761 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-da7da466-5d3d-4801-b4bc-873ff7856207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:58.036986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1453/2877] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:58.044962 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd606f-25aa-44a0-a7c7-651839ec6025 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:58.045249 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd606f-25aa-44a0-a7c7-651839ec6025 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:58.045519 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd606f-25aa-44a0-a7c7-651839ec6025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:58.065198 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a3cd606f-25aa-44a0-a7c7-651839ec6025 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:58.066569 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a3cd606f-25aa-44a0-a7c7-651839ec6025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:58.074686 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1425/2878] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:42:58.079360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7da466-5d3d-4801-b4bc-873ff7856207 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:58.089146 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7da466-5d3d-4801-b4bc-873ff7856207 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:58.095572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3e23-c5c6-4c55-8824-06e647b6a41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:58.096417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3e23-c5c6-4c55-8824-06e647b6a41b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:58.096689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3e23-c5c6-4c55-8824-06e647b6a41b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:58.096952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3e23-c5c6-4c55-8824-06e647b6a41b admin admin] PATH_INFO: `/v3/domains` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:58.097586 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3e23-c5c6-4c55-8824-06e647b6a41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:42:58.098138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3e23-c5c6-4c55-8824-06e647b6a41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:58.099545 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3e23-c5c6-4c55-8824-06e647b6a41b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:58.111374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1454/2879] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:42:58.115455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd606f-25aa-44a0-a7c7-651839ec6025 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:58.125055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd606f-25aa-44a0-a7c7-651839ec6025 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:58.129664 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52e793-b92e-4d58-be63-0ffc5356f1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:58.130279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52e793-b92e-4d58-be63-0ffc5356f1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:58.130560 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52e793-b92e-4d58-be63-0ffc5356f1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:58.130829 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52e793-b92e-4d58-be63-0ffc5356f1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:58.132454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52e793-b92e-4d58-be63-0ffc5356f1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:58.134370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52e793-b92e-4d58-be63-0ffc5356f1c0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:58.229959 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1426/2880] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:42:58 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:58.231122 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3e23-c5c6-4c55-8824-06e647b6a41b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:58.241273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3e23-c5c6-4c55-8824-06e647b6a41b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:58.297313 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f11cf5-3066-4dfa-9ef7-56e8ecde8bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:58.297940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f11cf5-3066-4dfa-9ef7-56e8ecde8bad admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:58.298238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f11cf5-3066-4dfa-9ef7-56e8ecde8bad admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:58.298508 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f11cf5-3066-4dfa-9ef7-56e8ecde8bad admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:58.299106 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f11cf5-3066-4dfa-9ef7-56e8ecde8bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:58.300995 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f11cf5-3066-4dfa-9ef7-56e8ecde8bad admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:58.404125 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1455/2881] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:42:58 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:42:58.404268 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52e793-b92e-4d58-be63-0ffc5356f1c0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:58.414197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52e793-b92e-4d58-be63-0ffc5356f1c0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:58.419549 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f619b-b695-445f-b960-449a7011b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:58.419802 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f619b-b695-445f-b960-449a7011b09c admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:58.420629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f619b-b695-445f-b960-449a7011b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:58.420968 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f619b-b695-445f-b960-449a7011b09c admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:58.421964 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f619b-b695-445f-b960-449a7011b09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:58.423626 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f619b-b695-445f-b960-449a7011b09c admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:58.435504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1427/2882] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:58.442939 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f11cf5-3066-4dfa-9ef7-56e8ecde8bad admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:58.454480 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f11cf5-3066-4dfa-9ef7-56e8ecde8bad admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:58.459320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823c3da-5d6b-4075-9c3a-4e43cf3684db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:58.459913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c3da-5d6b-4075-9c3a-4e43cf3684db admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:58.460188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c3da-5d6b-4075-9c3a-4e43cf3684db admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:58.460453 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c3da-5d6b-4075-9c3a-4e43cf3684db admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:58.461085 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823c3da-5d6b-4075-9c3a-4e43cf3684db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:58.462720 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823c3da-5d6b-4075-9c3a-4e43cf3684db admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:58.473768 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1456/2883] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:42:58.478183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f619b-b695-445f-b960-449a7011b09c admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:58.488520 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f619b-b695-445f-b960-449a7011b09c admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:58.493182 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093eb0e-500b-4f2c-906f-de578713969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:58.493853 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093eb0e-500b-4f2c-906f-de578713969e admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:58.494134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093eb0e-500b-4f2c-906f-de578713969e admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:58.494407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093eb0e-500b-4f2c-906f-de578713969e admin admin] PATH_INFO: `/v3/projects/8d10690c68d848109ee1507125710b9f/users/9aa2429d0e8f41a6a4f5c99e4dde3b36/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:58.510831 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093eb0e-500b-4f2c-906f-de578713969e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d10690c68d848109ee1507125710b9f, user_id=9aa2429d0e8f41a6a4f5c99e4dde3b36, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:58.513241 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093eb0e-500b-4f2c-906f-de578713969e admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:58.616264 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1428/2884] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:42:58 2022] PUT /identity/v3/projects/8d10690c68d848109ee1507125710b9f/users/9aa2429d0e8f41a6a4f5c99e4dde3b36/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:42:58.624141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f4f56-75a4-4e92-9cfa-0ff5cb20bd45 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:58.624141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f4f56-75a4-4e92-9cfa-0ff5cb20bd45 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:58.624141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f4f56-75a4-4e92-9cfa-0ff5cb20bd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:58.642504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-4d1f4f56-75a4-4e92-9cfa-0ff5cb20bd45 None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:42:58.646677 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4d1f4f56-75a4-4e92-9cfa-0ff5cb20bd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:58.693063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:58.705367 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1457/2885] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:42:58.715278 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093eb0e-500b-4f2c-906f-de578713969e admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:42:58.717513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-4093eb0e-500b-4f2c-906f-de578713969e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:42:58.757006 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:42:58.778378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093eb0e-500b-4f2c-906f-de578713969e admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:42:58.782530 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c236f0-cebb-4364-978e-20d5b8ea5421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:42:58.784378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c236f0-cebb-4364-978e-20d5b8ea5421 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:42:58.784378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c236f0-cebb-4364-978e-20d5b8ea5421 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:42:58.784378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c236f0-cebb-4364-978e-20d5b8ea5421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:42:58.792527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c236f0-cebb-4364-978e-20d5b8ea5421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:42:58.793963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c236f0-cebb-4364-978e-20d5b8ea5421 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:42:58.804364 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1429/2886] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:43:00.129127 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb0fd6-e16d-4093-8abe-3500e39bd7ad None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:00.129641 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb0fd6-e16d-4093-8abe-3500e39bd7ad None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:00.129740 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb0fd6-e16d-4093-8abe-3500e39bd7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:00.159555 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-efcb0fd6-e16d-4093-8abe-3500e39bd7ad None None] MFA Rules not processed for user `9aa2429d0e8f41a6a4f5c99e4dde3b36`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:00.164714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-efcb0fd6-e16d-4093-8abe-3500e39bd7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:00.228792 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1458/2887] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:00.233593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c236f0-cebb-4364-978e-20d5b8ea5421 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:00.243234 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c236f0-cebb-4364-978e-20d5b8ea5421 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:00.247705 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5315b42-e715-4ee4-80bb-114ecac7aa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:00.248311 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5315b42-e715-4ee4-80bb-114ecac7aa75 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:00.248589 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5315b42-e715-4ee4-80bb-114ecac7aa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:00.248856 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5315b42-e715-4ee4-80bb-114ecac7aa75 admin admin] PATH_INFO: `/v3/projects` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:00.249458 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5315b42-e715-4ee4-80bb-114ecac7aa75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:00.251374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5315b42-e715-4ee4-80bb-114ecac7aa75 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:00.396197 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1430/2888] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:43:00 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:43:00.400283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb0fd6-e16d-4093-8abe-3500e39bd7ad None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:00.412788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb0fd6-e16d-4093-8abe-3500e39bd7ad None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:00.470084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb405ac-9bac-46bb-b543-9e4ea8dca91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:00.470685 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb405ac-9bac-46bb-b543-9e4ea8dca91b admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:00.470963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb405ac-9bac-46bb-b543-9e4ea8dca91b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:00.471493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb405ac-9bac-46bb-b543-9e4ea8dca91b admin admin] PATH_INFO: `/v3/users` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:00.472108 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb405ac-9bac-46bb-b543-9e4ea8dca91b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:00.474056 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb405ac-9bac-46bb-b543-9e4ea8dca91b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:00.573974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1459/2889] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:43:00 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:43:00.582222 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5315b42-e715-4ee4-80bb-114ecac7aa75 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:00.593389 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5315b42-e715-4ee4-80bb-114ecac7aa75 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:00.598547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be308751-49cc-41a6-abc4-d216c95881da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:00.599142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be308751-49cc-41a6-abc4-d216c95881da admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:00.599415 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be308751-49cc-41a6-abc4-d216c95881da admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:00.599709 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be308751-49cc-41a6-abc4-d216c95881da admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:00.600346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be308751-49cc-41a6-abc4-d216c95881da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:00.602020 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be308751-49cc-41a6-abc4-d216c95881da admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:00.612682 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1431/2890] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:43:00.616891 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb405ac-9bac-46bb-b543-9e4ea8dca91b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:00.626931 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb405ac-9bac-46bb-b543-9e4ea8dca91b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:00.631455 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc02160-145c-42a7-bea5-628486090553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:00.632118 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc02160-145c-42a7-bea5-628486090553 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:00.632393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc02160-145c-42a7-bea5-628486090553 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:00.632657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc02160-145c-42a7-bea5-628486090553 admin admin] PATH_INFO: `/v3/projects/1121015187d743c7842246ad6783bced/users/6a85293f839643908a561d05d9c131d1/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:00.647380 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc02160-145c-42a7-bea5-628486090553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1121015187d743c7842246ad6783bced, user_id=6a85293f839643908a561d05d9c131d1, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:00.649998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc02160-145c-42a7-bea5-628486090553 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:00.723800 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1460/2891] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:43:00 2022] PUT /identity/v3/projects/1121015187d743c7842246ad6783bced/users/6a85293f839643908a561d05d9c131d1/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:00.727718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be308751-49cc-41a6-abc4-d216c95881da admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:00.737889 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be308751-49cc-41a6-abc4-d216c95881da admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:00.793629 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22364c54-cc7d-4290-a55e-3c3298a5f2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:00.794262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22364c54-cc7d-4290-a55e-3c3298a5f2eb admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:00.794544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22364c54-cc7d-4290-a55e-3c3298a5f2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:00.794818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22364c54-cc7d-4290-a55e-3c3298a5f2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:00.795453 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22364c54-cc7d-4290-a55e-3c3298a5f2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:00.797072 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22364c54-cc7d-4290-a55e-3c3298a5f2eb admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:00.806678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1432/2892] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:43:00.810816 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc02160-145c-42a7-bea5-628486090553 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:00.821195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc02160-145c-42a7-bea5-628486090553 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:00.825779 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd891f-c4c0-4535-af4e-85488efef2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:00.827058 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd891f-c4c0-4535-af4e-85488efef2e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:00.827292 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd891f-c4c0-4535-af4e-85488efef2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:00.827562 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd891f-c4c0-4535-af4e-85488efef2e2 admin admin] PATH_INFO: `/v3/projects/1121015187d743c7842246ad6783bced/users/6a85293f839643908a561d05d9c131d1/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:00.829551 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd891f-c4c0-4535-af4e-85488efef2e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1121015187d743c7842246ad6783bced, user_id=6a85293f839643908a561d05d9c131d1, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:00.832254 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd891f-c4c0-4535-af4e-85488efef2e2 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:00.843075 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1461/2893] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:43:00 2022] PUT /identity/v3/projects/1121015187d743c7842246ad6783bced/users/6a85293f839643908a561d05d9c131d1/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:02.223764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616cef4-b0e4-4635-a35e-684b9db3ac96 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:02.223764 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616cef4-b0e4-4635-a35e-684b9db3ac96 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:02.224386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616cef4-b0e4-4635-a35e-684b9db3ac96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:02.261666 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-c616cef4-b0e4-4635-a35e-684b9db3ac96 None None] MFA Rules not processed for user `6a85293f839643908a561d05d9c131d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:02.265127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c616cef4-b0e4-4635-a35e-684b9db3ac96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:02.338955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1433/2894] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:02.350747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd891f-c4c0-4535-af4e-85488efef2e2 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:02.352525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-f4fd891f-c4c0-4535-af4e-85488efef2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:02.393946 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:02.411188 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd891f-c4c0-4535-af4e-85488efef2e2 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:02.414802 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3d20f-9e16-4048-b980-bfe149c278f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:02.415407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3d20f-9e16-4048-b980-bfe149c278f0 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:02.415671 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3d20f-9e16-4048-b980-bfe149c278f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:02.415938 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3d20f-9e16-4048-b980-bfe149c278f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:02.472059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3d20f-9e16-4048-b980-bfe149c278f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:02.473481 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3d20f-9e16-4048-b980-bfe149c278f0 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:02.483252 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1462/2895] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:43:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:43:04.253975 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616cef4-b0e4-4635-a35e-684b9db3ac96 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:04.255431 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-c616cef4-b0e4-4635-a35e-684b9db3ac96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:04.268143 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616cef4-b0e4-4635-a35e-684b9db3ac96 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:04.271840 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eca31-4d01-495a-9f6a-776e82e83c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:04.272457 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eca31-4d01-495a-9f6a-776e82e83c5f service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:04.272720 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eca31-4d01-495a-9f6a-776e82e83c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:04.277588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eca31-4d01-495a-9f6a-776e82e83c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:04.282544 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2eca31-4d01-495a-9f6a-776e82e83c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:04.283969 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2eca31-4d01-495a-9f6a-776e82e83c5f service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:04.293542 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1434/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:43:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:43:12.705223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=32 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:14.289156 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:20.031300 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3d20f-9e16-4048-b980-bfe149c278f0 service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:20.033418 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-2da3d20f-9e16-4048-b980-bfe149c278f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:20.108600 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3d20f-9e16-4048-b980-bfe149c278f0 service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:20.113625 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b76d1-f42c-4f51-92bd-ae52b380ab8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:20.114256 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b76d1-f42c-4f51-92bd-ae52b380ab8c service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:20.114534 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b76d1-f42c-4f51-92bd-ae52b380ab8c service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:20.114804 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b76d1-f42c-4f51-92bd-ae52b380ab8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:20.116504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ad2b76d1-f42c-4f51-92bd-ae52b380ab8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:20.127604 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b76d1-f42c-4f51-92bd-ae52b380ab8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:20.129107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b76d1-f42c-4f51-92bd-ae52b380ab8c service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:20.138672 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1463/2897] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:43:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:43:20.174134 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eca31-4d01-495a-9f6a-776e82e83c5f service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:20.235279 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eca31-4d01-495a-9f6a-776e82e83c5f service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:20.239773 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:20.240378 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:20.240653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:20.240919 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] PATH_INFO: `/v3/users/9aa2429d0e8f41a6a4f5c99e4dde3b36` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:20.242030 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa2429d0e8f41a6a4f5c99e4dde3b36)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:20.243929 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:20.406890 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa2429d0e8f41a6a4f5c99e4dde3b36'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:20.491597 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa2429d0e8f41a6a4f5c99e4dde3b36'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:20.504557 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa2429d0e8f41a6a4f5c99e4dde3b36'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:20.511642 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa2429d0e8f41a6a4f5c99e4dde3b36'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:20.512277 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa2429d0e8f41a6a4f5c99e4dde3b36'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:20.518714 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1435/2898] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:43:20 2022] DELETE /identity/v3/users/9aa2429d0e8f41a6a4f5c99e4dde3b36 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:20.808860 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b76d1-f42c-4f51-92bd-ae52b380ab8c service placement] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:20.821699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b76d1-f42c-4f51-92bd-ae52b380ab8c service placement] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:20.880055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:20.880775 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:20.881054 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:20.881326 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] PATH_INFO: `/v3/projects/8d10690c68d848109ee1507125710b9f` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:20.882469 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d10690c68d848109ee1507125710b9f)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:20.884358 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:21.059107 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d10690c68d848109ee1507125710b9f', 'request_id': 'req-5e75820f-578a-43d5-8100-8266e09ebef0'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.065772 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d10690c68d848109ee1507125710b9f', 'request_id': 'req-5e75820f-578a-43d5-8100-8266e09ebef0'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.133491 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d10690c68d848109ee1507125710b9f', 'request_id': 'req-5e75820f-578a-43d5-8100-8266e09ebef0'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.144442 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] The token cache is being invalidate because project 8d10690c68d848109ee1507125710b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:43:21.144722 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.145329 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.146838 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1464/2899] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:43:20 2022] DELETE /identity/v3/projects/8d10690c68d848109ee1507125710b9f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:21.150715 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:21.152228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:21.166301 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ce082-31db-481b-8f20-160bb90ebe0b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:21.222375 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:21.223004 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:21.223245 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:21.223509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] PATH_INFO: `/v3/users/6a85293f839643908a561d05d9c131d1` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:21.224612 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a85293f839643908a561d05d9c131d1)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:21.226543 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:21.247994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:21.456845 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a85293f839643908a561d05d9c131d1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.527204 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a85293f839643908a561d05d9c131d1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.539938 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a85293f839643908a561d05d9c131d1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.546509 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a85293f839643908a561d05d9c131d1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.547127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-b65d5c42-ff92-4391-9399-117b9b53eac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a85293f839643908a561d05d9c131d1'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:21.552912 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1436/2900] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:43:21 2022] DELETE /identity/v3/users/6a85293f839643908a561d05d9c131d1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:21.791405 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:21.792813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:21.806091 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75820f-578a-43d5-8100-8266e09ebef0 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:21.822912 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:21.863357 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:21.863986 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:21.864266 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:21.864538 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] PATH_INFO: `/v3/projects/1121015187d743c7842246ad6783bced` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:21.865639 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1121015187d743c7842246ad6783bced)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:21.867614 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:22.019218 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1121015187d743c7842246ad6783bced', 'request_id': 'req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:22.025657 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1121015187d743c7842246ad6783bced', 'request_id': 'req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:22.115698 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1121015187d743c7842246ad6783bced', 'request_id': 'req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:22.126433 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] The token cache is being invalidate because project 1121015187d743c7842246ad6783bced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:43:22.126716 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:22.127328 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-fd06acb1-e87b-4a78-b2bd-7b59181fb6c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:22.128789 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1465/2901] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:43:21 2022] DELETE /identity/v3/projects/1121015187d743c7842246ad6783bced => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:22.133879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf248-3d71-40ef-b6a2-ea7bab2d6faa None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.134161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf248-3d71-40ef-b6a2-ea7bab2d6faa None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.136577 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf248-3d71-40ef-b6a2-ea7bab2d6faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.153915 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-150bf248-3d71-40ef-b6a2-ea7bab2d6faa None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:22.155096 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-150bf248-3d71-40ef-b6a2-ea7bab2d6faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:22.212854 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1437/2902] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:22.221066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793cad0-c5d9-48c3-9030-7a034f83bfcd None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.221356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793cad0-c5d9-48c3-9030-7a034f83bfcd None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.221631 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793cad0-c5d9-48c3-9030-7a034f83bfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.242560 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-c793cad0-c5d9-48c3-9030-7a034f83bfcd None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:22.243796 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c793cad0-c5d9-48c3-9030-7a034f83bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:22.252467 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1466/2903] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:22.256673 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bf248-3d71-40ef-b6a2-ea7bab2d6faa None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:22.266819 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bf248-3d71-40ef-b6a2-ea7bab2d6faa None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:22.271464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dade2-43f6-44cc-b78b-b4cf73eeaa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:22.272055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dade2-43f6-44cc-b78b-b4cf73eeaa95 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.272329 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dade2-43f6-44cc-b78b-b4cf73eeaa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.272596 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dade2-43f6-44cc-b78b-b4cf73eeaa95 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.273239 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dade2-43f6-44cc-b78b-b4cf73eeaa95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:43:22.273540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dade2-43f6-44cc-b78b-b4cf73eeaa95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:22.275165 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dade2-43f6-44cc-b78b-b4cf73eeaa95 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:22.286455 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1438/2904] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 17:43:22.290892 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793cad0-c5d9-48c3-9030-7a034f83bfcd None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:22.300776 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793cad0-c5d9-48c3-9030-7a034f83bfcd None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:22.305606 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe8cd5-0f73-49f8-bd68-8ee53760c541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:22.306247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe8cd5-0f73-49f8-bd68-8ee53760c541 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.306520 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe8cd5-0f73-49f8-bd68-8ee53760c541 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.306784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe8cd5-0f73-49f8-bd68-8ee53760c541 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.307407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe8cd5-0f73-49f8-bd68-8ee53760c541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:22.309353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe8cd5-0f73-49f8-bd68-8ee53760c541 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:22.437915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1467/2905] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:43:22 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:43:22.441891 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dade2-43f6-44cc-b78b-b4cf73eeaa95 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:22.451865 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dade2-43f6-44cc-b78b-b4cf73eeaa95 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:22.508407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2c3cc-92b1-419d-be47-9fefb502db6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:22.508993 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2c3cc-92b1-419d-be47-9fefb502db6b admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.509267 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2c3cc-92b1-419d-be47-9fefb502db6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.509532 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2c3cc-92b1-419d-be47-9fefb502db6b admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.510287 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2c3cc-92b1-419d-be47-9fefb502db6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:22.512180 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2c3cc-92b1-419d-be47-9fefb502db6b admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:22.624472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1439/2906] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:43:22 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:43:22.629243 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe8cd5-0f73-49f8-bd68-8ee53760c541 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:22.639651 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe8cd5-0f73-49f8-bd68-8ee53760c541 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:22.644374 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa45558-a17f-495d-a8d6-f78d9915362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:22.644994 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa45558-a17f-495d-a8d6-f78d9915362b admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.645273 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa45558-a17f-495d-a8d6-f78d9915362b admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.645541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa45558-a17f-495d-a8d6-f78d9915362b admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.646215 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa45558-a17f-495d-a8d6-f78d9915362b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:22.647803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa45558-a17f-495d-a8d6-f78d9915362b admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:22.658169 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1468/2907] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:43:22.662049 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2c3cc-92b1-419d-be47-9fefb502db6b admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:22.672411 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2c3cc-92b1-419d-be47-9fefb502db6b admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:22.676951 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16bc67-ae16-4846-be81-b1a25a14ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:22.677531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16bc67-ae16-4846-be81-b1a25a14ec86 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.677830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16bc67-ae16-4846-be81-b1a25a14ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.678104 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16bc67-ae16-4846-be81-b1a25a14ec86 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.678733 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16bc67-ae16-4846-be81-b1a25a14ec86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:22.680309 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16bc67-ae16-4846-be81-b1a25a14ec86 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:22.690754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1440/2908] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:43:22.694575 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa45558-a17f-495d-a8d6-f78d9915362b admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:22.704250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa45558-a17f-495d-a8d6-f78d9915362b admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:22.708763 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3429d82-8d87-4fe3-828c-7b959aaecfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:22.709393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3429d82-8d87-4fe3-828c-7b959aaecfdb admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.709667 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3429d82-8d87-4fe3-828c-7b959aaecfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.709974 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3429d82-8d87-4fe3-828c-7b959aaecfdb admin admin] PATH_INFO: `/v3/projects/a15a6a0bdaf542b2ab3b176680680043/users/06b837cb6f684da38f5e17054e72141d/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.725383 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3429d82-8d87-4fe3-828c-7b959aaecfdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15a6a0bdaf542b2ab3b176680680043, user_id=06b837cb6f684da38f5e17054e72141d, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:22.728049 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3429d82-8d87-4fe3-828c-7b959aaecfdb admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:22.772386 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1469/2909] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:43:22 2022] PUT /identity/v3/projects/a15a6a0bdaf542b2ab3b176680680043/users/06b837cb6f684da38f5e17054e72141d/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:43:22.791407 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fca32-6fb5-4471-8a53-2d200d2bd31c None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.791689 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fca32-6fb5-4471-8a53-2d200d2bd31c None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.791957 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fca32-6fb5-4471-8a53-2d200d2bd31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.800628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:22.819479 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-556fca32-6fb5-4471-8a53-2d200d2bd31c None None] MFA Rules not processed for user `06b837cb6f684da38f5e17054e72141d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:22.820678 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-556fca32-6fb5-4471-8a53-2d200d2bd31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:22.884902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1441/2910] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:22.895518 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3429d82-8d87-4fe3-828c-7b959aaecfdb admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:22.897233 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-a3429d82-8d87-4fe3-828c-7b959aaecfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:22.947909 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:22.952953 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3429d82-8d87-4fe3-828c-7b959aaecfdb admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:22.956643 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02abd2-92af-4ec3-94c1-80fca25075c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:22.957231 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02abd2-92af-4ec3-94c1-80fca25075c8 service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:22.957504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02abd2-92af-4ec3-94c1-80fca25075c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:22.960301 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02abd2-92af-4ec3-94c1-80fca25075c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:22.965889 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02abd2-92af-4ec3-94c1-80fca25075c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:22.967288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02abd2-92af-4ec3-94c1-80fca25075c8 service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:22.976584 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1470/2911] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:43:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:43:32.689884 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fca32-6fb5-4471-8a53-2d200d2bd31c None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:32.751842 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fca32-6fb5-4471-8a53-2d200d2bd31c None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:32.756614 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:32.757228 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:32.757502 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:32.757763 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] PATH_INFO: `/v3/users/06b837cb6f684da38f5e17054e72141d` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:32.758879 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b837cb6f684da38f5e17054e72141d)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:32.760851 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:32.950902 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b837cb6f684da38f5e17054e72141d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.024129 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b837cb6f684da38f5e17054e72141d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.036383 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b837cb6f684da38f5e17054e72141d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.043169 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b837cb6f684da38f5e17054e72141d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.043821 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b837cb6f684da38f5e17054e72141d'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.047278 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=5 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:33.050055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1442/2912] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:43:32 2022] DELETE /identity/v3/users/06b837cb6f684da38f5e17054e72141d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:33.054729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55c7b3-74ce-4dea-82a6-2910a05a727b None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:33.055038 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55c7b3-74ce-4dea-82a6-2910a05a727b None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:33.055407 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55c7b3-74ce-4dea-82a6-2910a05a727b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:33.074514 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-6d55c7b3-74ce-4dea-82a6-2910a05a727b None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:33.077984 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6d55c7b3-74ce-4dea-82a6-2910a05a727b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:33.135371 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1471/2913] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:33.146045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:33.147470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:33.161290 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:33.203045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0ac28-f49a-4c00-b08d-156edfa159d7 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:33.206628 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28447168-dbcd-47cb-a5f4-cead6832e51a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:33.207818 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28447168-dbcd-47cb-a5f4-cead6832e51a service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:33.208088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28447168-dbcd-47cb-a5f4-cead6832e51a service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:33.208349 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28447168-dbcd-47cb-a5f4-cead6832e51a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:33.217153 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28447168-dbcd-47cb-a5f4-cead6832e51a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:33.218976 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28447168-dbcd-47cb-a5f4-cead6832e51a service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:33.228988 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1443/2914] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:43:33.632293 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55c7b3-74ce-4dea-82a6-2910a05a727b None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:33.633737 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-6d55c7b3-74ce-4dea-82a6-2910a05a727b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:33.646052 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55c7b3-74ce-4dea-82a6-2910a05a727b None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:33.650729 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:33.653765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:33.653765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:33.653765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] PATH_INFO: `/v3/projects/a15a6a0bdaf542b2ab3b176680680043` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:33.653765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15a6a0bdaf542b2ab3b176680680043)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:33.655021 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:33.767461 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15a6a0bdaf542b2ab3b176680680043', 'request_id': 'req-1e64640e-bc64-4844-a5f7-8732b8f7d06c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.773348 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15a6a0bdaf542b2ab3b176680680043', 'request_id': 'req-1e64640e-bc64-4844-a5f7-8732b8f7d06c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.851633 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15a6a0bdaf542b2ab3b176680680043', 'request_id': 'req-1e64640e-bc64-4844-a5f7-8732b8f7d06c'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.862340 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] The token cache is being invalidate because project a15a6a0bdaf542b2ab3b176680680043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:43:33.862624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.863863 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-1e64640e-bc64-4844-a5f7-8732b8f7d06c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:33.865580 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1472/2915] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:43:33 2022] DELETE /identity/v3/projects/a15a6a0bdaf542b2ab3b176680680043 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:43:33.870464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cc311-282c-4cc1-a3ca-37bf731eb45e None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:33.870910 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cc311-282c-4cc1-a3ca-37bf731eb45e None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:33.871031 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cc311-282c-4cc1-a3ca-37bf731eb45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:33.890322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-a29cc311-282c-4cc1-a3ca-37bf731eb45e None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:33.891518 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-a29cc311-282c-4cc1-a3ca-37bf731eb45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:33.949702 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1444/2916] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:33.957572 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db58d-dffb-4431-9c84-1cc581a5624a None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:33.958276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db58d-dffb-4431-9c84-1cc581a5624a None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:33.958276 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db58d-dffb-4431-9c84-1cc581a5624a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:33.977911 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-fc3db58d-dffb-4431-9c84-1cc581a5624a None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:33.979102 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-fc3db58d-dffb-4431-9c84-1cc581a5624a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:33.986814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1473/2917] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:43:33.991445 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cc311-282c-4cc1-a3ca-37bf731eb45e None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:34.001060 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cc311-282c-4cc1-a3ca-37bf731eb45e None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:34.005616 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff47534-05f5-4f4b-a301-fbbae6148f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:34.006240 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff47534-05f5-4f4b-a301-fbbae6148f98 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:34.006505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff47534-05f5-4f4b-a301-fbbae6148f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:34.006791 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff47534-05f5-4f4b-a301-fbbae6148f98 admin admin] PATH_INFO: `/v3/domains` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:34.007436 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff47534-05f5-4f4b-a301-fbbae6148f98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 17:43:34.007734 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff47534-05f5-4f4b-a301-fbbae6148f98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:34.009312 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff47534-05f5-4f4b-a301-fbbae6148f98 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:34.021374 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1445/2918] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Aug 8 17:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 17:43:34.025288 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3db58d-dffb-4431-9c84-1cc581a5624a None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:34.034595 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3db58d-dffb-4431-9c84-1cc581a5624a None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:34.039831 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f64b13-be4a-42b3-a2c9-22fdfa5a6b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:34.040420 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f64b13-be4a-42b3-a2c9-22fdfa5a6b01 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:34.040695 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f64b13-be4a-42b3-a2c9-22fdfa5a6b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:34.040964 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f64b13-be4a-42b3-a2c9-22fdfa5a6b01 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:34.041589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f64b13-be4a-42b3-a2c9-22fdfa5a6b01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:34.043548 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f64b13-be4a-42b3-a2c9-22fdfa5a6b01 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:34.150811 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1474/2919] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:43:34 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:43:34.154926 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff47534-05f5-4f4b-a301-fbbae6148f98 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:34.165504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff47534-05f5-4f4b-a301-fbbae6148f98 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:34.219052 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a3246-685e-408c-8e8e-f7c2eff66113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:34.219639 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a3246-685e-408c-8e8e-f7c2eff66113 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:34.219911 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a3246-685e-408c-8e8e-f7c2eff66113 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:34.220177 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a3246-685e-408c-8e8e-f7c2eff66113 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:34.221033 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a3246-685e-408c-8e8e-f7c2eff66113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:34.223593 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a3246-685e-408c-8e8e-f7c2eff66113 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:34.313150 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1446/2920] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:43:34 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:43:34.314304 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f64b13-be4a-42b3-a2c9-22fdfa5a6b01 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:34.324659 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f64b13-be4a-42b3-a2c9-22fdfa5a6b01 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:34.329247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1cb7f-efc2-4011-826a-1e230aadfa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:34.329873 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1cb7f-efc2-4011-826a-1e230aadfa0c admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:34.330154 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1cb7f-efc2-4011-826a-1e230aadfa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:34.330431 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1cb7f-efc2-4011-826a-1e230aadfa0c admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:34.331066 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1cb7f-efc2-4011-826a-1e230aadfa0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:34.333141 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1cb7f-efc2-4011-826a-1e230aadfa0c admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:34.342913 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1475/2921] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:43:34.346888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a3246-685e-408c-8e8e-f7c2eff66113 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:34.357199 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a3246-685e-408c-8e8e-f7c2eff66113 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:34.361667 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48626d9-63a5-44e0-ae51-df1666fddc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:34.362273 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48626d9-63a5-44e0-ae51-df1666fddc7f admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:34.362547 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48626d9-63a5-44e0-ae51-df1666fddc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:34.362808 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48626d9-63a5-44e0-ae51-df1666fddc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:34.363454 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48626d9-63a5-44e0-ae51-df1666fddc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:34.365047 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48626d9-63a5-44e0-ae51-df1666fddc7f admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:34.375212 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1447/2922] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:43:34.379095 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1cb7f-efc2-4011-826a-1e230aadfa0c admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:34.389045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1cb7f-efc2-4011-826a-1e230aadfa0c admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:34.393624 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e40cc-2b6d-4042-86f2-3e2f7bb60b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:34.394283 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e40cc-2b6d-4042-86f2-3e2f7bb60b40 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:34.395059 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e40cc-2b6d-4042-86f2-3e2f7bb60b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:34.395332 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e40cc-2b6d-4042-86f2-3e2f7bb60b40 admin admin] PATH_INFO: `/v3/projects/78ea264ce6bf4a0cbfc96f3eafb540b9/users/3364ca1eb9f94b3c9e638ca26c9ffdd5/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:34.409493 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e40cc-2b6d-4042-86f2-3e2f7bb60b40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ea264ce6bf4a0cbfc96f3eafb540b9, user_id=3364ca1eb9f94b3c9e638ca26c9ffdd5, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:34.412799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e40cc-2b6d-4042-86f2-3e2f7bb60b40 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:34.498963 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1476/2923] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:43:34 2022] PUT /identity/v3/projects/78ea264ce6bf4a0cbfc96f3eafb540b9/users/3364ca1eb9f94b3c9e638ca26c9ffdd5/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:43:34.509749 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a407a-2786-4fb6-924b-3d0389eead13 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:34.510583 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a407a-2786-4fb6-924b-3d0389eead13 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:34.510863 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a407a-2786-4fb6-924b-3d0389eead13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:34.543272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-687a407a-2786-4fb6-924b-3d0389eead13 None None] MFA Rules not processed for user `3364ca1eb9f94b3c9e638ca26c9ffdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:34.544913 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-687a407a-2786-4fb6-924b-3d0389eead13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:34.554161 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:34.610524 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1448/2924] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:34.615184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e40cc-2b6d-4042-86f2-3e2f7bb60b40 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:34.636103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:34.678291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e40cc-2b6d-4042-86f2-3e2f7bb60b40 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:34.683035 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78089e1-ac67-4ce5-b3f3-6ac58c8c7226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:34.683684 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78089e1-ac67-4ce5-b3f3-6ac58c8c7226 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:34.683973 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78089e1-ac67-4ce5-b3f3-6ac58c8c7226 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:34.684247 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78089e1-ac67-4ce5-b3f3-6ac58c8c7226 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:34.684878 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78089e1-ac67-4ce5-b3f3-6ac58c8c7226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:34.686845 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78089e1-ac67-4ce5-b3f3-6ac58c8c7226 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:34.820747 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1477/2925] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:43:34 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 17:43:34.824830 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a407a-2786-4fb6-924b-3d0389eead13 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:34.836242 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a407a-2786-4fb6-924b-3d0389eead13 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:34.891582 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50837c24-ca30-48c0-9328-677b2dfc6514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:34.892183 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50837c24-ca30-48c0-9328-677b2dfc6514 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:34.892385 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50837c24-ca30-48c0-9328-677b2dfc6514 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:34.892733 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50837c24-ca30-48c0-9328-677b2dfc6514 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:34.893352 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50837c24-ca30-48c0-9328-677b2dfc6514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:34.895971 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50837c24-ca30-48c0-9328-677b2dfc6514 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:34.984736 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1449/2926] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:43:34 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:43:34.989353 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78089e1-ac67-4ce5-b3f3-6ac58c8c7226 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:34.999641 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78089e1-ac67-4ce5-b3f3-6ac58c8c7226 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:35.005291 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5d902-d9eb-4ecc-94c5-fa16d6266a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:35.005983 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5d902-d9eb-4ecc-94c5-fa16d6266a4f admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.006263 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5d902-d9eb-4ecc-94c5-fa16d6266a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.006563 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5d902-d9eb-4ecc-94c5-fa16d6266a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.007200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5d902-d9eb-4ecc-94c5-fa16d6266a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:35.008828 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5d902-d9eb-4ecc-94c5-fa16d6266a4f admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:35.021013 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1478/2927] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:43:35.024985 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50837c24-ca30-48c0-9328-677b2dfc6514 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:35.035365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50837c24-ca30-48c0-9328-677b2dfc6514 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:35.040227 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec0195-61aa-4028-a990-6ffa9c008745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:35.040810 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec0195-61aa-4028-a990-6ffa9c008745 admin admin] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.041089 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec0195-61aa-4028-a990-6ffa9c008745 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.041422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec0195-61aa-4028-a990-6ffa9c008745 admin admin] PATH_INFO: `/v3/roles` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.042110 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec0195-61aa-4028-a990-6ffa9c008745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:35.043703 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec0195-61aa-4028-a990-6ffa9c008745 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:35.054317 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1450/2928] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 17:43:35.058581 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5d902-d9eb-4ecc-94c5-fa16d6266a4f admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:35.069193 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5d902-d9eb-4ecc-94c5-fa16d6266a4f admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:35.073837 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd249df3-a95f-41b3-89d9-99f86703b298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:35.074490 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd249df3-a95f-41b3-89d9-99f86703b298 admin admin] REQUEST_METHOD: `PUT` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.074770 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd249df3-a95f-41b3-89d9-99f86703b298 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.075045 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd249df3-a95f-41b3-89d9-99f86703b298 admin admin] PATH_INFO: `/v3/projects/8dfad070aa654f5bb21f8750cd98527c/users/54c3eaa7237d4762b0abb14d9b9cb6cc/roles/d01bf1c8bfca4691affaddcd9487c751` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.090648 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd249df3-a95f-41b3-89d9-99f86703b298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dfad070aa654f5bb21f8750cd98527c, user_id=54c3eaa7237d4762b0abb14d9b9cb6cc, role_id=d01bf1c8bfca4691affaddcd9487c751)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:35.093320 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd249df3-a95f-41b3-89d9-99f86703b298 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:35.166472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1479/2929] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:43:35 2022] PUT /identity/v3/projects/8dfad070aa654f5bb21f8750cd98527c/users/54c3eaa7237d4762b0abb14d9b9cb6cc/roles/d01bf1c8bfca4691affaddcd9487c751 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:35.177955 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd926-aa44-407b-87de-5a8be889b537 None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.178263 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd926-aa44-407b-87de-5a8be889b537 None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.178527 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fd926-aa44-407b-87de-5a8be889b537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.207045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-1b8fd926-aa44-407b-87de-5a8be889b537 None None] MFA Rules not processed for user `54c3eaa7237d4762b0abb14d9b9cb6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:35.208295 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1b8fd926-aa44-407b-87de-5a8be889b537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:35.271470 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1451/2930] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 17:43:35.275932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd249df3-a95f-41b3-89d9-99f86703b298 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:35.336788 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd249df3-a95f-41b3-89d9-99f86703b298 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:35.341409 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012f9b-5b83-4755-86f2-3e55c28e9e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:35.342039 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012f9b-5b83-4755-86f2-3e55c28e9e38 admin admin] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.342319 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012f9b-5b83-4755-86f2-3e55c28e9e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.342589 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012f9b-5b83-4755-86f2-3e55c28e9e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.343203 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74012f9b-5b83-4755-86f2-3e55c28e9e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:35.345183 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74012f9b-5b83-4755-86f2-3e55c28e9e38 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:35.482647 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1480/2931] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Aug 8 17:43:35 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:43:35.486648 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fd926-aa44-407b-87de-5a8be889b537 None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:35.498503 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fd926-aa44-407b-87de-5a8be889b537 None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:35.517531 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:35.559584 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b68a9-b722-4945-b948-0f0534b982b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:35.560464 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b68a9-b722-4945-b948-0f0534b982b3 admin admin] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.560826 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b68a9-b722-4945-b948-0f0534b982b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.561185 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b68a9-b722-4945-b948-0f0534b982b3 admin admin] PATH_INFO: `/v3/users` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.562065 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b68a9-b722-4945-b948-0f0534b982b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:35.564252 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b68a9-b722-4945-b948-0f0534b982b3 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:35.647868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1452/2932] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Aug 8 17:43:35 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 17:43:35.651879 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012f9b-5b83-4755-86f2-3e55c28e9e38 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:35.662885 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012f9b-5b83-4755-86f2-3e55c28e9e38 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:35.667393 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22affc6b-6339-432d-bf06-268be9444864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:35.667981 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22affc6b-6339-432d-bf06-268be9444864 admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.668258 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22affc6b-6339-432d-bf06-268be9444864 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.668583 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22affc6b-6339-432d-bf06-268be9444864 admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.669221 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22affc6b-6339-432d-bf06-268be9444864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:35.670829 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22affc6b-6339-432d-bf06-268be9444864 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:35.683799 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1481/2933] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:43:35.687795 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b68a9-b722-4945-b948-0f0534b982b3 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:35.698282 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b68a9-b722-4945-b948-0f0534b982b3 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:35.703019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3426d-ac96-4a4b-86e1-e551eb750dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:35.703668 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3426d-ac96-4a4b-86e1-e551eb750dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.703946 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3426d-ac96-4a4b-86e1-e551eb750dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.704214 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3426d-ac96-4a4b-86e1-e551eb750dc3 admin admin] PATH_INFO: `/v3/projects/33b69002b9fa4089841526833f852445/users/20fa29b775724f65a9df962a5267c101/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.719731 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3426d-ac96-4a4b-86e1-e551eb750dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b69002b9fa4089841526833f852445, user_id=20fa29b775724f65a9df962a5267c101, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:35.723314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3426d-ac96-4a4b-86e1-e551eb750dc3 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:35.830540 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1453/2934] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:43:35 2022] PUT /identity/v3/projects/33b69002b9fa4089841526833f852445/users/20fa29b775724f65a9df962a5267c101/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:35.835023 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22affc6b-6339-432d-bf06-268be9444864 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:35.845248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22affc6b-6339-432d-bf06-268be9444864 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:35.886755 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:35.902040 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea69947-f27a-4155-b3e9-3bb7da39278d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:35.902638 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea69947-f27a-4155-b3e9-3bb7da39278d admin admin] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.902948 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea69947-f27a-4155-b3e9-3bb7da39278d admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.903235 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea69947-f27a-4155-b3e9-3bb7da39278d admin admin] PATH_INFO: `/v3/roles` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.903864 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea69947-f27a-4155-b3e9-3bb7da39278d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:35.905446 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea69947-f27a-4155-b3e9-3bb7da39278d admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:35.915261 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1482/2935] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Aug 8 17:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 17:43:35.922581 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3426d-ac96-4a4b-86e1-e551eb750dc3 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:35.930645 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3426d-ac96-4a4b-86e1-e551eb750dc3 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:35.935496 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d99a2-0e76-4535-b49a-531a24d0a8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:35.936190 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d99a2-0e76-4535-b49a-531a24d0a8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.936472 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d99a2-0e76-4535-b49a-531a24d0a8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.936746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d99a2-0e76-4535-b49a-531a24d0a8e0 admin admin] PATH_INFO: `/v3/projects/33b69002b9fa4089841526833f852445/users/20fa29b775724f65a9df962a5267c101/roles/dac864cdcb1a4cc283d63a7f33d20d17` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.938756 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d99a2-0e76-4535-b49a-531a24d0a8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b69002b9fa4089841526833f852445, user_id=20fa29b775724f65a9df962a5267c101, role_id=dac864cdcb1a4cc283d63a7f33d20d17)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:35.942055 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d99a2-0e76-4535-b49a-531a24d0a8e0 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:35.954146 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1454/2936] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Aug 8 17:43:35 2022] PUT /identity/v3/projects/33b69002b9fa4089841526833f852445/users/20fa29b775724f65a9df962a5267c101/roles/dac864cdcb1a4cc283d63a7f33d20d17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:35.964323 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624718a7-3ebf-4140-bcaf-561cc7bf87b7 None None] REQUEST_METHOD: `POST` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:35.964610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624718a7-3ebf-4140-bcaf-561cc7bf87b7 None None] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:35.964884 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624718a7-3ebf-4140-bcaf-561cc7bf87b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:35.992512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.auth.core [None req-624718a7-3ebf-4140-bcaf-561cc7bf87b7 None None] MFA Rules not processed for user `20fa29b775724f65a9df962a5267c101`. Rule list: `[]` (Enabled: `True`). {{(pid=92261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:35.993710 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-624718a7-3ebf-4140-bcaf-561cc7bf87b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:36.063366 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1483/2937] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:36.074850 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d99a2-0e76-4535-b49a-531a24d0a8e0 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:36.076548 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-1c4d99a2-0e76-4535-b49a-531a24d0a8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:36.134680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d99a2-0e76-4535-b49a-531a24d0a8e0 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:36.138162 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eded2da-ff1d-40f5-9e7f-ee379f0c8456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:36.138746 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eded2da-ff1d-40f5-9e7f-ee379f0c8456 service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:36.139018 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eded2da-ff1d-40f5-9e7f-ee379f0c8456 service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:36.139280 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eded2da-ff1d-40f5-9e7f-ee379f0c8456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:36.195903 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eded2da-ff1d-40f5-9e7f-ee379f0c8456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:36.197344 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eded2da-ff1d-40f5-9e7f-ee379f0c8456 service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:36.206860 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1455/2938] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:43:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:43:38.199610 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624718a7-3ebf-4140-bcaf-561cc7bf87b7 None None] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:38.201163 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-624718a7-3ebf-4140-bcaf-561cc7bf87b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:38.213413 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624718a7-3ebf-4140-bcaf-561cc7bf87b7 None None] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:38.216869 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f0d77-ff2b-422f-9ef7-0899964358ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:38.217504 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f0d77-ff2b-422f-9ef7-0899964358ce service cinder] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:38.217732 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f0d77-ff2b-422f-9ef7-0899964358ce service cinder] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:38.218032 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f0d77-ff2b-422f-9ef7-0899964358ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:38.273696 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f0d77-ff2b-422f-9ef7-0899964358ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:38.275167 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f0d77-ff2b-422f-9ef7-0899964358ce service cinder] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:38.283743 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1484/2939] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:43:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:43:39.584793 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eded2da-ff1d-40f5-9e7f-ee379f0c8456 service cinder] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:39.593591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eded2da-ff1d-40f5-9e7f-ee379f0c8456 service cinder] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:39.597451 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e1454-227b-42c0-9bb1-a4a84c54981b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e860aad24688491c9a89ddfb35922eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:39.598680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e1454-227b-42c0-9bb1-a4a84c54981b service cinder] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:39.599066 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e1454-227b-42c0-9bb1-a4a84c54981b service cinder] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:39.599368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e1454-227b-42c0-9bb1-a4a84c54981b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:39.608622 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e1454-227b-42c0-9bb1-a4a84c54981b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:39.610142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e1454-227b-42c0-9bb1-a4a84c54981b service cinder] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:39.619386 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1456/2940] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:43:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 17:43:44.202084 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f0d77-ff2b-422f-9ef7-0899964358ce service cinder] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:44.263063 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f0d77-ff2b-422f-9ef7-0899964358ce service cinder] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:44.267485 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:44.268365 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:44.268650 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:44.268919 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] PATH_INFO: `/v3/users/3364ca1eb9f94b3c9e638ca26c9ffdd5` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:44.270055 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3364ca1eb9f94b3c9e638ca26c9ffdd5)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:44.271970 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:44.276424 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=100 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:44.465653 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3364ca1eb9f94b3c9e638ca26c9ffdd5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:44.577998 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3364ca1eb9f94b3c9e638ca26c9ffdd5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:44.585540 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3364ca1eb9f94b3c9e638ca26c9ffdd5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:44.598250 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3364ca1eb9f94b3c9e638ca26c9ffdd5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:44.598935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3364ca1eb9f94b3c9e638ca26c9ffdd5'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:44.604915 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1485/2941] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:43:44 2022] DELETE /identity/v3/users/3364ca1eb9f94b3c9e638ca26c9ffdd5 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:44.609365 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c65947-d48e-4682-9890-26220cae74bc None None] REQUEST_METHOD: `POST` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:44.609984 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c65947-d48e-4682-9890-26220cae74bc None None] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:44.610088 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c65947-d48e-4682-9890-26220cae74bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:44.629299 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.auth.core [None req-58c65947-d48e-4682-9890-26220cae74bc None None] MFA Rules not processed for user `8502ad6c48f34f22831d3b9a1d5e8e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 17:43:44.630523 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-58c65947-d48e-4682-9890-26220cae74bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:44.638456 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:44.688572 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1457/2942] 10.222.0.57 () {58 vars in 958 bytes} [Mon Aug 8 17:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 17:43:44.698360 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:44.701425 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:44.761298 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c84da8-bdcd-4525-90c1-0c574d8edc40 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:44.764896 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0345b-df0e-405d-bad2-029ff6e719e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:44.765541 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0345b-df0e-405d-bad2-029ff6e719e3 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:44.765868 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0345b-df0e-405d-bad2-029ff6e719e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:44.766134 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0345b-df0e-405d-bad2-029ff6e719e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:44.775699 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0345b-df0e-405d-bad2-029ff6e719e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:44.777171 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0345b-df0e-405d-bad2-029ff6e719e3 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:44.787051 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1486/2943] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:43:45.197304 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c65947-d48e-4682-9890-26220cae74bc None None] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:45.198777 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-58c65947-d48e-4682-9890-26220cae74bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:45.212493 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c65947-d48e-4682-9890-26220cae74bc None None] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:45.217504 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:45.219505 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:45.219790 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:45.220034 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] PATH_INFO: `/v3/projects/78ea264ce6bf4a0cbfc96f3eafb540b9` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:45.221743 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ea264ce6bf4a0cbfc96f3eafb540b9)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:45.223717 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:45.344109 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ea264ce6bf4a0cbfc96f3eafb540b9', 'request_id': 'req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.350422 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ea264ce6bf4a0cbfc96f3eafb540b9', 'request_id': 'req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.423948 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ea264ce6bf4a0cbfc96f3eafb540b9', 'request_id': 'req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.434967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] The token cache is being invalidate because project 78ea264ce6bf4a0cbfc96f3eafb540b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:43:45.435252 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.435880 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.437340 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1458/2944] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:43:45 2022] DELETE /identity/v3/projects/78ea264ce6bf4a0cbfc96f3eafb540b9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:45.444050 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0345b-df0e-405d-bad2-029ff6e719e3 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:45.454488 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0345b-df0e-405d-bad2-029ff6e719e3 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:45.515132 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:45.515765 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:45.516044 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:45.516318 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] PATH_INFO: `/v3/users/54c3eaa7237d4762b0abb14d9b9cb6cc` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:45.517445 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54c3eaa7237d4762b0abb14d9b9cb6cc)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:45.519417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:45.603714 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:45.716784 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54c3eaa7237d4762b0abb14d9b9cb6cc'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.818229 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54c3eaa7237d4762b0abb14d9b9cb6cc'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.825440 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54c3eaa7237d4762b0abb14d9b9cb6cc'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.839270 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54c3eaa7237d4762b0abb14d9b9cb6cc'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.839949 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54c3eaa7237d4762b0abb14d9b9cb6cc'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:45.846041 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1487/2945] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:43:45 2022] DELETE /identity/v3/users/54c3eaa7237d4762b0abb14d9b9cb6cc => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:46.276063 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:46.277491 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:46.291334 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ab669-6b92-4cd9-a1f2-8dcd66ff0d81 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:46.346868 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:46.348097 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:46.348370 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:46.348637 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] PATH_INFO: `/v3/projects/8dfad070aa654f5bb21f8750cd98527c` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:46.349727 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dfad070aa654f5bb21f8750cd98527c)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:46.351633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:46.486356 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:46.493186 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dfad070aa654f5bb21f8750cd98527c', 'request_id': 'req-7012915c-d61b-462d-990e-0ff7daeef7ee'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.499254 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dfad070aa654f5bb21f8750cd98527c', 'request_id': 'req-7012915c-d61b-462d-990e-0ff7daeef7ee'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.647045 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dfad070aa654f5bb21f8750cd98527c', 'request_id': 'req-7012915c-d61b-462d-990e-0ff7daeef7ee'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.658160 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] The token cache is being invalidate because project 8dfad070aa654f5bb21f8750cd98527c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:43:46.658696 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.659685 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.661210 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1459/2946] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:43:46 2022] DELETE /identity/v3/projects/8dfad070aa654f5bb21f8750cd98527c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:46.665211 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:46.674682 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:46.679238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130d03d-8d4b-435b-aa4c-5702ddbdceb4 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:46.735725 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:46.736356 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:46.736680 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:46.736990 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] PATH_INFO: `/v3/users/20fa29b775724f65a9df962a5267c101` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:46.738138 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fa29b775724f65a9df962a5267c101)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:46.740048 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:46.871814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fa29b775724f65a9df962a5267c101'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.920814 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fa29b775724f65a9df962a5267c101'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.928248 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fa29b775724f65a9df962a5267c101'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.939723 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fa29b775724f65a9df962a5267c101'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.940352 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fa29b775724f65a9df962a5267c101'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:46.946689 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1488/2947] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:43:46 2022] DELETE /identity/v3/users/20fa29b775724f65a9df962a5267c101 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:43:47.344257 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:43:47.346844 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:43:47.366934 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012915c-d61b-462d-990e-0ff7daeef7ee admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:43:47.424337 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:43:47.424967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:43:47.425246 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:43:47.425514 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] PATH_INFO: `/v3/projects/33b69002b9fa4089841526833f852445` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:43:47.426711 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33b69002b9fa4089841526833f852445)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:43:47.429142 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:43:47.556633 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b69002b9fa4089841526833f852445', 'request_id': 'req-d1e1021e-8207-45a6-a12d-1b505da9df22'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:47.562788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b69002b9fa4089841526833f852445', 'request_id': 'req-d1e1021e-8207-45a6-a12d-1b505da9df22'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:47.641094 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b69002b9fa4089841526833f852445', 'request_id': 'req-d1e1021e-8207-45a6-a12d-1b505da9df22'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:47.652679 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] The token cache is being invalidate because project 33b69002b9fa4089841526833f852445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:43:47.652967 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:47.653613 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:43:47.655147 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1460/2948] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:43:47 2022] DELETE /identity/v3/projects/33b69002b9fa4089841526833f852445 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 17:43:56.943251 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=72,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:43:57.649322 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=61,keystone:DELETE=11,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:44:34.853902 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:44:34.855325 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:34.925607 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5514d9b-cf89-43a2-9cec-29fbcf495280 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:44:34.984282 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11022f-d76d-4d10-a9d0-409ce4af9931 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:44:34.984890 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11022f-d76d-4d10-a9d0-409ce4af9931 service placement] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:44:34.985174 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11022f-d76d-4d10-a9d0-409ce4af9931 service placement] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:44:34.985444 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11022f-d76d-4d10-a9d0-409ce4af9931 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:44:34.987200 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-ae11022f-d76d-4d10-a9d0-409ce4af9931 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:34.998443 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11022f-d76d-4d10-a9d0-409ce4af9931 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:44:35.000030 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11022f-d76d-4d10-a9d0-409ce4af9931 service placement] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:44:35.009952 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1489/2949] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:44:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:44:36.613718 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:44:36.615307 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:36.689922 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1021e-8207-45a6-a12d-1b505da9df22 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:44:36.695255 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:44:36.695852 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:44:36.696125 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:44:36.696492 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:44:36.698262 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:36.711320 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:44:36.712905 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:44:36.725888 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1461/2950] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:44:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:44:39.196179 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11022f-d76d-4d10-a9d0-409ce4af9931 service placement] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:44:39.209940 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11022f-d76d-4d10-a9d0-409ce4af9931 service placement] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:44:39.215558 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b05cbc41b844d8932ba4c15d960289', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:44:39.219553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:44:39.219553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:44:39.219553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:44:39.219553 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:39.232170 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:44:39.236805 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:44:39.244315 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1490/2951] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:44:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:44:46.212416 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:44:46.213892 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:46.273019 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dd093-8b50-44ff-a23e-aee203286e9e service nova] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:44:46.276710 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:44:46.277314 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:44:46.277591 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:44:46.277898 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:44:46.280474 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:46.290368 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:44:46.291788 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:44:46.299958 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1462/2952] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:44:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:44:49.239735 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=64 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:44:51.723582 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:44:51.724926 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:51.736416 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c6e11-9a49-4c36-89bf-e9a64e59376d service nova] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:44:51.739935 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bc5e2-f210-4b9e-a3c9-a41491193780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f50ee57e2418417cbe68915ca3f00f76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:44:51.740525 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bc5e2-f210-4b9e-a3c9-a41491193780 service neutron] REQUEST_METHOD: `GET` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:44:51.740803 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bc5e2-f210-4b9e-a3c9-a41491193780 service neutron] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:44:51.741074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bc5e2-f210-4b9e-a3c9-a41491193780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:44:51.742757 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-b63bc5e2-f210-4b9e-a3c9-a41491193780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:51.827103 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bc5e2-f210-4b9e-a3c9-a41491193780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:44:51.829195 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bc5e2-f210-4b9e-a3c9-a41491193780 service neutron] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:44:51.848854 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1491/2953] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:44:56.296272 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=60 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:44:58.978611 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:44:58.980229 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:58.994567 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8d87c-acbb-4226-ad7e-cdef41d19cc6 service neutron] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:44:58.999037 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:44:58.999653 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:44:58.999930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:44:59.000199 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] PATH_INFO: `/v3/users/57662358169d4830b2ebb7d594efb701` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:44:59.001313 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] RBAC: Authorizing `identity:delete_user(user_id=57662358169d4830b2ebb7d594efb701)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:44:59.003258 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:44:59.164057 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57662358169d4830b2ebb7d594efb701'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.222346 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57662358169d4830b2ebb7d594efb701'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.235253 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57662358169d4830b2ebb7d594efb701'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.242276 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57662358169d4830b2ebb7d594efb701'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.242963 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57662358169d4830b2ebb7d594efb701'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.248925 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1463/2954] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:44:58 2022] DELETE /identity/v3/users/57662358169d4830b2ebb7d594efb701 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:44:59.509361 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bc5e2-f210-4b9e-a3c9-a41491193780 service neutron] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:44:59.520074 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bc5e2-f210-4b9e-a3c9-a41491193780 service neutron] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:44:59.576573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:44:59.577223 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:44:59.578156 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:44:59.578438 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] PATH_INFO: `/v3/projects/0f857768418e43d8bdbab85be767b896` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:44:59.579601 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f857768418e43d8bdbab85be767b896)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:44:59.581500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:44:59.697402 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f857768418e43d8bdbab85be767b896', 'request_id': 'req-4084b309-2557-48f7-91ca-15fa6d72c351'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.703472 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f857768418e43d8bdbab85be767b896', 'request_id': 'req-4084b309-2557-48f7-91ca-15fa6d72c351'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.805228 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f857768418e43d8bdbab85be767b896', 'request_id': 'req-4084b309-2557-48f7-91ca-15fa6d72c351'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.810585 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:44:59.817417 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] The token cache is being invalidate because project 0f857768418e43d8bdbab85be767b896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:44:59.819573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.819573 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:44:59.819916 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1492/2955] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:44:59 2022] DELETE /identity/v3/projects/0f857768418e43d8bdbab85be767b896 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:44:59.823972 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:44:59.825357 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:44:59.838434 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd280a-c550-406e-9c7f-c4f892c6f16a admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:44:59.895513 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:44:59.896135 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:44:59.896408 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:44:59.896680 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] PATH_INFO: `/v3/users/82b7005da8f04eedab6a32f6dc6484d4` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:44:59.897779 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b7005da8f04eedab6a32f6dc6484d4)` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:44:59.899737 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:45:00.058588 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b7005da8f04eedab6a32f6dc6484d4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.113498 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b7005da8f04eedab6a32f6dc6484d4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.125814 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b7005da8f04eedab6a32f6dc6484d4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.127754 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:45:00.133192 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b7005da8f04eedab6a32f6dc6484d4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.133930 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.notifications [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b7005da8f04eedab6a32f6dc6484d4'} {{(pid=92260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.139741 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1464/2956] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Aug 8 17:44:59 2022] DELETE /identity/v3/users/82b7005da8f04eedab6a32f6dc6484d4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:45:00.361932 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] Authenticating user token {{(pid=92261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:45:00.363395 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.fernet_utils [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:45:00.376886 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084b309-2557-48f7-91ca-15fa6d72c351 admin admin] Validating token access rules against request {{(pid=92261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:45:00.432856 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8502ad6c48f34f22831d3b9a1d5e8e5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0b6d65db9ee4a2cb85f51ab2f1fbc92', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:45:00.433500 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:45:00.433905 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] SCRIPT_NAME: `/identity` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:45:00.434184 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] PATH_INFO: `/v3/projects/3bca2c32913f43ad86326629903c5b4b` {{(pid=92261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:45:00.435274 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bca2c32913f43ad86326629903c5b4b)` {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:45:00.437238 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] RBAC: Authorization granted {{(pid=92261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:45:00.513093 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bca2c32913f43ad86326629903c5b4b', 'request_id': 'req-eafe7e2e-e283-4eab-bcbd-53b826af8936'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.519813 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bca2c32913f43ad86326629903c5b4b', 'request_id': 'req-eafe7e2e-e283-4eab-bcbd-53b826af8936'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.582474 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bca2c32913f43ad86326629903c5b4b', 'request_id': 'req-eafe7e2e-e283-4eab-bcbd-53b826af8936'} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.594226 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] The token cache is being invalidate because project 3bca2c32913f43ad86326629903c5b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 17:45:00.594512 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.595199 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG keystone.notifications [None req-eafe7e2e-e283-4eab-bcbd-53b826af8936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 17:45:00.596618 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: [pid: 92261|app: 0|req: 1493/2957] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Aug 8 17:45:00 2022] DELETE /identity/v3/projects/3bca2c32913f43ad86326629903c5b4b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 17:45:10.251155 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=4 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:45:10.590160 dv-cinder-852210-3-iscsi devstack@keystone.service[92261]: DEBUG dbcounter [-] [92261] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 17:49:38.872927 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] Authenticating user token {{(pid=92260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 17:49:38.874377 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:49:38.992000 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff581f-a8d0-4438-acce-3ec5942ce042 admin admin] Validating token access rules against request {{(pid=92260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 17:49:39.050867 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb3fe0-55ba-41d5-a0de-b928e8ea50d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5853f9b60cf34380b218a5cda94f587d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c1078a85a3648d08fb0fc64519bdfd3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 17:49:39.051471 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb3fe0-55ba-41d5-a0de-b928e8ea50d0 service placement] REQUEST_METHOD: `GET` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 17:49:39.051757 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb3fe0-55ba-41d5-a0de-b928e8ea50d0 service placement] SCRIPT_NAME: `/identity` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 17:49:39.052029 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb3fe0-55ba-41d5-a0de-b928e8ea50d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 17:49:39.053772 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.fernet_utils [None req-ecdb3fe0-55ba-41d5-a0de-b928e8ea50d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 17:49:39.065388 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb3fe0-55ba-41d5-a0de-b928e8ea50d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 17:49:39.066994 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb3fe0-55ba-41d5-a0de-b928e8ea50d0 service placement] RBAC: Authorization granted {{(pid=92260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 17:49:39.077405 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: [pid: 92260|app: 0|req: 1465/2958] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Aug 8 17:49:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 17:49:49.073127 dv-cinder-852210-3-iscsi devstack@keystone.service[92260]: DEBUG dbcounter [-] [92260] Writing DB stats keystone:SELECT=80 {{(pid=92260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}